A method for processing a patch against, related apparatus and storage medium

Abstract

Embodiments of the present application relate to the technical field of computer vision, and provide an adversarial patch processing method, related devices and a storage medium. The method comprises: obtaining a candidate adversarial patch, and superimposing the candidate adversarial patch on an initial image comprising a target object to obtain a candidate adversarial image; performing feature extraction on the candidate adversarial image to obtain adversarial image features of the candidate adversarial image; calculating a feature loss value between the adversarial image features and initial image features of the initial image to obtain a target loss value; if the target loss value does not satisfy a preset convergence condition, updating the candidate adversarial patch based on the target loss value to obtain an updated patch, and taking the updated patch as the candidate adversarial patch until the feature loss value satisfies the preset convergence condition, thereby obtaining a target adversarial patch. The present application can update the candidate adversarial patch to obtain an excellent target adversarial patch, thereby significantly improving the universality of the target adversarial patch and providing a better experience for users.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically to an anti-patch processing method, related apparatus, and storage medium. The related apparatus includes an anti-patch processing device, a computer device, a computer program product, and a chip system. The storage medium is a computer-readable storage medium. Background Technology

[0002] Image processing neural network models have wide applications across various fields. For example, pedestrian detection models are crucial in computer vision, used to detect pedestrians in images or videos and determine their location. Pedestrian detection models are widely used in artificial intelligence systems, driver assistance systems, intelligent robots, intelligent video surveillance, human behavior analysis, and intelligent transportation. However, an attack on a pedestrian detection model can have a significant impact on the entire system using it.

[0003] Existing technologies often employ attack algorithms to generate adversarial patches for performance evaluation of image processing neural network models. However, since current attack algorithms are developed specifically for particular neural network models, this means that attack algorithms cannot be shared between different image processing neural network models. Consequently, adversarial patches generated by the same attack algorithm cannot be used across different image processing neural network models, resulting in poor universality of adversarial patches.

[0004] In summary, existing technologies suffer from poor versatility in countermeasures against patches. Summary of the Invention

[0005] This application provides an adversarial patch processing method, related apparatus, and storage medium, which can optimize candidate adversarial patches, demonstrate the ability to test any image processing neural network model, and better improve the versatility of target adversarial patches.

[0006] In a first aspect, embodiments of this application provide a method for handling anti-patch measures, the method comprising:

[0007] Obtain candidate adversarial patches and overlay them onto the initial image including the target object to obtain a candidate adversarial image;

[0008] Feature extraction is performed on the candidate adversarial image to obtain the adversarial image features of the candidate adversarial image. The adversarial image features do not include the object features of the target object in the initial image, or they include some features of the target object in the initial image.

[0009] Calculate the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value;

[0010] If the target loss value does not meet the preset convergence condition, the candidate adversary patch is updated based on the target loss value to obtain the updated patch, and the updated patch is used as the candidate adversary patch until the target loss value meets the preset convergence condition to obtain the target adversary patch.

[0011] Secondly, embodiments of this application provide an anti-patch processing apparatus, which has functions corresponding to the anti-patch processing method provided in the first aspect above. The functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware.

[0012] In one embodiment, the anti-patch processing apparatus includes:

[0013] The input / output module is configured to acquire candidate adversarial patches and overlay the candidate adversarial patches with an initial image including the target object to obtain a candidate adversarial image;

[0014] The processing module is also configured to extract features from the candidate adversarial image to obtain adversarial image features of the candidate adversarial image. The adversarial image features do not include object features of the target object in the initial image, or include some features of the target object in the initial image.

[0015] The processing module is also configured to calculate the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value;

[0016] The processing module is also configured to update the candidate adversary patch based on the target loss value if the target loss value does not meet the preset convergence condition, obtain the updated patch, and use the updated patch as the candidate adversary patch until the target loss value meets the preset convergence condition and the target adversary patch is obtained.

[0017] In some embodiments, the processing module is specifically configured to fuse the adversarial image features and the initial image features to obtain fused image features; based on the fused image features, determine the feature loss value of the candidate adversarial image, and based on the feature loss value, determine the target loss value.

[0018] In some application embodiments, the processing module is specifically configured to identify adversarial patches in candidate adversarial images, and calculate the printing loss value of the adversarial patches based on the adversarial patches, wherein the printing loss value characterizes the color printing loss value and / or texture printing loss value of the adversarial patches; and generate a target loss value based on the printing loss value and the feature loss value.

[0019] In some embodiments, the processing module is specifically configured to calculate the smoothing loss value of the adversarial patch, which is obtained based on the feature distance between adjacent pixels in the adversarial patch; and generate a target loss value based on the smoothing loss value, the printing loss value, and the feature loss value.

[0020] In some application embodiments, the processing module is specifically configured to obtain the weights corresponding to the smoothing loss value, the printing loss value, and the feature loss value respectively; and to fuse the smoothing loss value, the printing loss value, the feature loss value, and the weights to obtain the target loss value.

[0021] In some embodiments, the processing module is specifically configured to update the candidate adversarial patch according to the target loss value to obtain an initial updated patch; and to trim the initial updated patch to obtain an updated patch.

[0022] In some embodiments, the processing module is specifically configured to obtain target pixels whose pixel values ​​are not within a preset range in the initial update patch; and to crop the target pixels to obtain the update patch.

[0023] In some application embodiments, the processing module is specifically configured to call a preset feature extraction model to classify objects in the candidate adversarial image to obtain object categories; obtain the actual object category of the object; and calculate the adversarial image features corresponding to the candidate adversarial image based on the object category and the actual object category.

[0024] In some embodiments, the processing module is specifically configured to calculate the category loss value between the object category and the actual object category; and to map the category loss value to obtain the adversarial image features corresponding to the candidate adversarial image.

[0025] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the anti-patch processing method as described in the first aspect.

[0026] Fourthly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the anti-patch processing method of the first aspect.

[0027] Fifthly, embodiments of this application provide a chip that includes a processor coupled to a transceiver of a terminal device, for executing the technical solution provided in the first aspect of embodiments of this application.

[0028] In a sixth aspect, embodiments of this application provide a chip system including a processor for supporting a terminal device in implementing the functions involved in the first aspect above, such as generating or processing information involved in the anti-patch processing method provided in the first aspect above.

[0029] In one possible design, the aforementioned chip system also includes a memory for storing program instructions and data necessary for the terminal. The chip system can be composed of chips or may include chips and other discrete components.

[0030] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a computer, cause the computer to execute the anti-patch processing method provided in the first aspect.

[0031] Compared to existing technologies, in this embodiment, a candidate adversarial patch is superimposed on an initial image including the target object to obtain a candidate adversarial image. Then, the adversarial image features extracted from the candidate adversarial image and the initial image features of the initial image can be used to calculate the target loss value so as to update the candidate adversarial patch based on the target loss value and obtain an excellent target adversarial patch. Because this application can overlay the candidate adversarial patch with the initial image to obtain a candidate adversarial image, when extracting features from the candidate adversarial image, it can obtain adversarial image features that either do not contain the target object or weaken some features of the target object. Thus, the feature loss value between the adversarial image features with similar features and the initial image features can be calculated to obtain a target loss value. Based on this target loss value, the candidate adversarial patch can be optimized to obtain a target adversarial patch. This allows the target adversarial patch to weaken or eliminate the features of candidate objects in any image. Furthermore, since any image processing neural network model extracts similar image features from the same image containing candidate objects (i.e., extracting features that either do not contain candidate objects or contain only some features of candidate objects), any image processing neural network model can fail to identify candidate objects in the image. This results in a target adversarial patch with excellent and universality. Instead of providing a unique adversarial patch for each image processing neural network model as in the prior art, the embodiments of this application improve the universality of the target adversarial patch compared to the prior art. Because the target adversarial patch of this application has higher versatility, the target adversarial patch obtained in the embodiments of this application can achieve ideal model testing results and can effectively test any image processing neural network model. Attached Figure Description

[0032] The objectives, features, and advantages of the embodiments of this application will become readily understood by referring to the accompanying drawings and the detailed description of the embodiments. Wherein:

[0033] Figure 1 This is a schematic diagram of an anti-patch processing system according to an embodiment of the anti-patch processing method in this application;

[0034] Figure 2 This is a flowchart illustrating an anti-patch processing method according to an embodiment of this application;

[0035] Figure 3 This is another flowchart illustrating the anti-patch processing method according to an embodiment of this application;

[0036] Figure 4 This is a schematic diagram illustrating the test results of the model according to an embodiment of this application;

[0037] Figure 5 Two figures illustrating the test results of the model according to an embodiment of this application;

[0038] Figure 6 This is a schematic diagram of the anti-patch processing device according to an embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the structure of a computer device according to an embodiment of this application;

[0040] Figure 8 This is a schematic diagram of the structure of a mobile phone in one embodiment of this application;

[0041] Figure 9 This is a schematic diagram of a server structure in one embodiment of this application.

[0042] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0043] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division; in actual applications, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be omitted or not performed. Additionally, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, indirect coupling between modules, or electrical or other similar forms of communication connection, none of which are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0044] This application also provides an adversarial patch processing method, related apparatus, and storage medium, applicable to adversarial patch processing systems in scenarios that improve the versatility of adversarial patches. The adversarial patch processing system may include a feature extraction device and a patch updating device. The feature extraction device and the patch updating device can be deployed integratedly or separately. The feature extraction device is at least used to acquire candidate adversarial patches, overlay the candidate adversarial patches with an initial image including a target object to obtain a candidate adversarial image, and extract features from the candidate adversarial image to obtain adversarial image features of the candidate adversarial image. The patch updating device may at least be used to calculate a feature loss value between the adversarial image features and the initial image features of the initial image to obtain a target loss value, and update the candidate adversarial patch based on the target loss value to obtain the target adversarial patch. The feature extraction device can be an application capable of acquiring candidate adversarial patches, overlaying the candidate adversarial patches with an initial image including a target object to obtain a candidate adversarial image, and extracting features from the candidate adversarial image to obtain adversarial image features of the candidate adversarial image; or it can be a server or terminal device that has an application installed capable of acquiring candidate adversarial patches, overlaying the candidate adversarial patches with an initial image including a target object to obtain a candidate adversarial image, and extracting features from the candidate adversarial image to obtain adversarial image features of the candidate adversarial image. The patch updating device can be an application capable of calculating a feature loss value between the adversarial image features and the initial image features of the initial image to obtain a target loss value, and updating the candidate adversarial patch based on the target loss value to obtain a target adversarial patch; or it can be a terminal device that has an application deployed capable of calculating a feature loss value between the adversarial image features and the initial image features of the initial image to obtain a target loss value, and updating the candidate adversarial patch based on the target loss value to obtain a target adversarial patch.

[0045] The solutions provided in this application involve technologies such as Artificial Intelligence (AI), Computer Vision (CV), and Machine Learning (ML), which are specifically illustrated through the following embodiments:

[0046] AI, or Artificial Intelligence, refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, Artificial Intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a manner similar to human intelligence. Artificial Intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0047] AI technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0048] Existing technologies often employ attack algorithms to generate adversarial patches for performance evaluation of image processing neural network models. However, because current attack algorithms are developed specifically for particular neural network models, they cannot be shared between different image processing neural network models. Consequently, adversarial patches generated by the same attack algorithm cannot be used across different image processing neural network models, meaning their versatility is poor. In other words, existing technologies suffer from poor versatility in adversarial patches.

[0049] Compared to existing technologies, in this embodiment, candidate adversarial patches can be obtained and superimposed on an initial image including the target object to obtain a candidate adversarial image; feature extraction is performed on the candidate adversarial image to obtain adversarial image features of the candidate adversarial image, wherein the adversarial image features do not include object features of the target object in the initial image, or include some features of the target object in the initial image; the feature loss value between the adversarial image features and the initial image features of the initial image is calculated to obtain a target loss value; if the target loss value does not meet the preset convergence condition, the candidate adversarial patch is updated based on the target loss value to obtain an updated patch, and the updated patch is used as a candidate adversarial patch until the target loss value meets the preset convergence condition to obtain the target adversarial patch. Since the embodiments of this application can utilize candidate adversarial patches to scramble target objects in candidate adversarial images, when extracting features from candidate adversarial images, adversarial image features that do not include object features of the target object in the initial image, or adversarial image features that include some features of the target object in the initial image, can be obtained. This allows the candidate adversarial image to be distinguished from the initial image, and the target loss value can be calculated based on the adversarial image features and the initial image features of the initial image. The candidate adversarial patch can then be optimized based on the target loss value to obtain an excellent and universal target adversarial patch.

[0050] In some implementations, the feature extraction device and the patch update device are deployed separately, as shown in the following figure. Figure 1 The anti-patch processing method provided in this application embodiment can be based on Figure 1 The image processing system shown is an implementation of an anti-patch processing system. This system may include a server 01 and a terminal device 02.

[0051] The server 01 may be a feature extraction device, in which a feature extraction program for extracting features from candidate adversarial images may be deployed.

[0052] The terminal device 02 may be a patch update device, which may be equipped with a patch update program for updating candidate adversaries and candidate adversary patches.

[0053] The server 01 can send the adversarial image features extracted from the candidate adversarial image to the terminal device 02; the terminal device 02 can receive the adversarial image features, calculate the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value, and then update the candidate adversarial patch based on the target loss value to obtain the target adversarial patch.

[0054] It should be noted that the server involved in the embodiments of this application can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0055] The terminal devices involved in the embodiments of this application can be devices that provide voice and / or data connectivity to users, handheld devices with wireless connectivity, or other processing devices connected to a wireless modem. Examples include mobile phones (or "cellular" phones) and computers with mobile terminals, such as portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted mobile devices that exchange voice and / or data with a wireless access network. Examples include Personal Communication Service (PCS) phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, Personal Digital Assistants (PDAs), and other similar devices.

[0056] Reference Figure 2 , Figure 2This is a flowchart illustrating an adversarial patch processing method provided in an embodiment of this application. The method can be executed by an adversarial patch processing device and can be applied to adversarial patch processing scenarios requiring improved versarial patch versatility. This application can utilize candidate adversarial patches to disrupt target objects in candidate adversarial images. Then, during feature extraction of the candidate adversarial images, adversarial image features that do not include object features of the target object in the initial image, or adversarial image features that include some features of the target object in the initial image, can be obtained. Thus, a target loss value can be calculated based on the adversarial image features and the initial image features of the initial image, allowing for optimization of the candidate adversarial patch based on the target loss value, resulting in an excellent and versatile target adversarial patch.

[0057] The method includes steps 101-104:

[0058] Step 101: Obtain candidate adversarial patches and overlay them with the initial image including the target object to obtain a candidate adversarial image.

[0059] In this context, a candidate adversarial patch can refer to an adversarial patch capable of disrupting the target object in the initial image. For example, a candidate adversarial patch can be characterized as a colored cardboard patch.

[0060] The target object can be a pedestrian, an animal, an object, etc. For example, an animal could be a cat, an object could be a vehicle, and so on.

[0061] In this context, a candidate adversarial image can refer to an image that uses candidate adversarial patches as interference factors and can be used to attack image processing neural network models. For example, the content of a candidate object image can include scenery or pedestrians obscured by colored cardboard patches.

[0062] This application can first obtain candidate adversarial patches as interference factors, and then use the interference factors to perturb the initial image in order to obtain a candidate adversarial image with interference factors. For example, this application can superimpose the candidate adversarial patch with the initial image including the target object to obtain a candidate adversarial image.

[0063] In order to improve the generalization of candidate adversarial images, this application can process candidate adversarial patches. Specifically, in step 101, the step of "overlaying the candidate adversarial patch with the initial image including the target object to obtain a candidate adversarial image" can be specifically: transforming the candidate adversarial patch to obtain a transformed adversarial patch; overlaying the transformed adversarial patch with the initial image to obtain a candidate adversarial image.

[0064] It is understood that the transformation of candidate adversarial patches in this application is to simulate various interference factors in order to improve the generalization of candidate adversarial patches, thereby improving the generalization of candidate adversarial images. Among these interference factors, the main ones are the effects of camera noise and illumination.

[0065] The method of "transforming the candidate adversarial patch to obtain the transformed adversarial patch" can be as follows: obtain the transformation parameters, and transform the candidate adversarial patch according to the transformation parameters to obtain the transformed adversarial patch. Specifically, there are various ways to transform the candidate adversarial patch, such as random Gaussian noise, brightness transformation, contrast transformation, etc.; correspondingly, the transformation parameters can include random Gaussian noise parameters corresponding to random Gaussian noise, brightness transformation parameters corresponding to brightness transformation, and contrast transformation parameters corresponding to contrast transformation.

[0066] After obtaining the transformed adversarial patch, this application can overlay the transformed adversarial patch onto the initial image to obtain a candidate adversarial image. It is understood that this application can overlay the transformed adversarial patch onto the initial image, and the overlay position of the transformed adversarial patch on the initial image is not explicitly limited in principle. When this application is applied to pedestrian detection attack scenarios, the preferred overlay position of the transformed adversarial patch is the image region in the initial image where the target object is located, such as the image region where the pedestrian is located, to facilitate subsequent improvement of the effectiveness and robustness of the target adversarial patch. Based on this, specifically, the step "overlaying the transformed adversarial patch onto the initial image to obtain a candidate adversarial image" can be performed as follows: identifying the image region where the target object is located in the initial image; masking the image region with the transformed adversarial patch; and then overlaying the transformed adversarial patch onto the initial image to obtain a candidate adversarial image.

[0067] In step 101, the method for the step "obtaining candidate adversarial patches" can be: extracting the initial image from the cloud or a local database.

[0068] Step 102: Extract features from the candidate adversarial images to obtain the adversarial image features of the candidate adversarial images.

[0069] Adversarial image features can refer to the features obtained by feature extraction from candidate adversarial images. Adversarial image features do not include object features of the target object in the initial image, or they may include some features of the target object in the initial image.

[0070] The adversarial image features can be represented as a gradient attention map. Each pixel in the gradient attention map has a corresponding numerical value, and each numerical value represents the contribution of the corresponding pixel to the classification result obtained by the candidate adversarial image. The larger the numerical value, the higher the contribution to the classification result. The following details the acquisition of the gradient attention map.

[0071] After obtaining the candidate adversarial image, this application can extract the adversarial image features of the candidate adversarial image to optimize the candidate adversarial patch based on the adversarial image features. Specifically, in step 102, the method of "extracting features from the candidate adversarial image to obtain the adversarial image features of the candidate adversarial image" can be as follows: calling a preset feature extraction model to classify the objects in the candidate adversarial image to obtain the object category; obtaining the actual object category of the object; and calculating the adversarial image features corresponding to the candidate adversarial image based on the object category and the actual object category.

[0072] The preset feature extraction model can be an image processing model. This image processing model can be an alternative model to the preset image processing model that the adversarial patch is to test. The alternative model can refer to a neural network model that can achieve the same or similar functions as the preset image processing model.

[0073] This application utilizes an alternative model to obtain adversarial image features of candidate adversarial images, enabling attacks on the preset image processing model after obtaining a superior target adversarial patch. Specifically, the step "calling the preset feature extraction model to classify objects in the candidate adversarial image and obtain object categories" can be implemented as follows: calling the preset feature extraction model to identify object regions in the candidate adversarial image; using the preset feature extraction model to extract features from the object regions to obtain object region features; and classifying the objects in the candidate adversarial image based on the object region features to obtain object categories.

[0074] Here, "object" can refer to an object with candidate adversarial patches. For example, an object can be the object obtained after a target object is overlaid with candidate adversarial patches. The actual object category of the object can be carried by the candidate adversarial image.

[0075] After obtaining the object category, this application can use the object category to obtain the adversarial image features corresponding to the candidate object image. Specifically, the method of "calculating the adversarial image features corresponding to the candidate adversarial image based on the object category and the actual object category" can be as follows: calculate the category loss value between the object category and the actual object category; map the category loss value to obtain the adversarial image features corresponding to the candidate adversarial image.

[0076] The step "calculate the category loss value between the object category and the actual object category" can be performed by obtaining the category loss function and calculating the category loss value between the object category and the actual object category. The category loss function is shown in formula (1).

[0077] loss=l_cls(f(x,w),c) Calculation formula (1)

[0078] Here, `loss` can refer to the class loss value; `c` can refer to the actual object class; `f(x,w)` can refer to the object class; `f()` can refer to the preset feature extraction model; `x` can refer to the candidate adversarial image; `w` can refer to the weights of the preset feature extraction model; and `l_cls()` can refer to the class loss function. Specifically, the class loss function can be the logarithmic loss function, the MSE squared loss function, etc.

[0079] After calculating the class loss using the class loss function, this application can use the class loss value to obtain the adversarial image features. Specifically, the step "mapping the class loss value to obtain the adversarial image features corresponding to the candidate adversarial image" can be done as follows: obtaining the feature mapping function; mapping the class loss value according to the feature mapping function to obtain the adversarial image features corresponding to the candidate adversarial image. The feature mapping function can be as shown in formula (2):

[0080]

[0081] Here, att can refer to adversarial image features; ReLU() can refer to an activation function. It can refer to the partial derivative of the class loss function with respect to the candidate adversarial image.

[0082] It is understandable here that adversarial image features, namely gradient attention maps, are affected by noise and may have negative effects in some locations. Therefore, the parts of the gradient attention map that are less than 0 are filtered out before calculating the gradient attention loss. Hence, the ReLU() operation is added to the formula.

[0083] In addition, before step 101, this application may also acquire an initial image and extract features from the initial image to obtain the initial image features corresponding to the initial image, so that the initial image features can be used to calculate the feature loss value in step 103.

[0084] In order to facilitate the attack on the preset image processing model after obtaining excellent target adversarial patches, this application can use the aforementioned preset feature extraction model to process the initial image. Specifically, the step of "extracting features from the initial image to obtain the initial image features corresponding to the initial image" can be done as follows: calling the preset feature extraction model to classify the target objects in the initial image to obtain the target object category; obtaining the target actual object category of the target object; and calculating the initial image features corresponding to the initial image based on the target object category and the target actual object category.

[0085] The step "calling the preset feature extraction model to classify the target objects in the initial image and obtain the target object category" can be found in the description of "calling the preset feature extraction model to classify the objects in the candidate adversarial image and obtain the object category" above, and will not be repeated here.

[0086] The step "Calculate the initial image features corresponding to the initial image based on the target object category and the actual target object category" can be found in the description of "Calculate the adversarial image features corresponding to the candidate adversarial image based on the object category and the actual object category" above, and will not be repeated here.

[0087] It should be noted that the initial image features can also be represented as gradient attention maps. For details on the gradient attention maps corresponding to the initial image features, please refer to the aforementioned description of the gradient attention maps corresponding to the adversarial image features. These details will not be repeated here.

[0088] Before elaborating on steps 103 and 104, it is understood that the main advantage of this application lies in exploring the common features of different image processing models. Specifically, for the same candidate image containing a candidate object, why can different image processing models detect the candidate object, and what common features do they share, despite the significant differences in their detection algorithms? Through extensive experimental trials, this application has found that the gradient attention maps of different image processing models have extremely high similarity, which leads to the fact that different image processing models can detect the same candidate object.

[0089] Based on this, this application can use gradient attention maps to update candidate adversarial patches to obtain target adversarial patches. In this way, when testing a preset image processing model, the target adversarial patch is superimposed on the test image including the candidate objects to obtain a test image; then, the preset image processing model corresponding to the test image is used for testing. The target adversarial patch perturbs the candidate objects in the test image, causing the candidate objects to weaken or even disappear in the gradient attention map corresponding to the test image, thus preventing the preset image processing model from detecting the objects.

[0090] It should be noted that the target adversarial patch of this application can be used to test not only white-box models but also black-box models. For example, when the preset image processing model is a black-box model, because the gradient attention maps of different image processing models are similar, the black-box model cannot identify the candidate objects in the test image when using the test image generated based on the target adversarial patch. The generation process of the target adversarial patch is explained in detail below through steps 103 to 104.

[0091] Step 103: Calculate the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value.

[0092] After extracting the adversarial image features, this application can use the adversarial image features to obtain the target loss value, so that the candidate adversarial image patch can be updated using the target loss value. Specifically, the method of "calculating the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value" can be as shown in steps 31 and 32:

[0093] Step 31: Fuse the adversarial image features and the initial image features to obtain the fused image features.

[0094] The feature loss value represents the loss between the initial image and the candidate adversarial image.

[0095] In step 31, the step of "fusing adversarial image features and initial image features to obtain fused image features" can be performed by: calculating the target feature distance between the adversarial image features and the initial image features, and using the target feature distance as the fused image features.

[0096] Step 32: Based on the features of the fused image, determine the feature loss value of the candidate adversarial image, and based on the feature loss value, determine the target loss value.

[0097] After obtaining the fused image features, this application can use the fused image features to determine the feature loss value. Specifically, in step 32, the method of "determining the feature loss value of the candidate adversarial image based on the fused image features" can be: using a feature loss function to fuse the initial image features and the adversarial image features to obtain the feature loss value of the candidate adversarial image. The feature loss function can be found in formula (3):

[0098] l adv (att adv ,att ori )=||att adv ||2-||att adv -att ori ||2 Formula (3)

[0099] Among them, l adv (att adv ,att ori ) can represent the feature loss value; at ori It can represent the initial image features, att adv This can represent adversarial image features. The first term of equation (3) is ||att adv ||2 means to minimize the magnitude of the adversarial image feature, even to the point of being close to 0; the second term of formula (3) is ||att adv -att ori ||2 represents the distance between the adversarial image features and the initial image features. Since this application aims to make this distance as large as possible, the loss here is the subtraction of this distance.

[0100] Among them, the features of the fused image can be used as the feature loss value.

[0101] After obtaining the feature loss value, this application can determine the target loss value so as to update the candidate adversarial patch based on the target loss value. There are several ways to perform the step "determine the target loss value based on the feature loss value." For example, the feature loss value can be used as the target loss value; or the feature loss value can be combined with other loss values ​​to obtain the target loss value, as shown in steps 321 to 322.

[0102] Step 321: Identify adversarial patches in the candidate adversarial images and calculate the printing loss value of the adversarial patches based on them.

[0103] Among them, the adversarial patch can refer to the patch corresponding to the candidate adversarial image after the candidate adversarial patch is superimposed on the initial image to obtain the candidate adversarial image.

[0104] The printing loss value represents the color printing loss value and / or texture printing loss value of the adversarial patch. The adversarial patch can be the same as the candidate adversarial patch.

[0105] After identifying the adversarial patch in the candidate adversarial image, this application can use the adversarial patch to calculate the printing loss value in order to improve the attack performance of the target adversarial patch. In step 321, the method of "calculating the printing loss value of the adversarial patch according to the adversarial patch" can be: obtaining the printing parameters of the adversarial patch; performing mapping processing on the printing parameters to obtain the printing loss value of the adversarial patch.

[0106] One approach is to use a print parameter calculation function to calculate the print parameters of the adversarial patch based on the pixel values ​​of the adversarial patch.

[0107] One approach is to use a printing loss function to map the printing parameters and obtain the printing loss value of the anti-patch.

[0108] The printing loss function can be found in formula (4):

[0109]

[0110] Among them, l p This could refer to the printing loss value; p print This could refer to the printing parameters of an adversarial patch, which characterize the degree to which an adversarial patch cannot be printed; c print It can refer to the color parameters that the printer can print.

[0111] It is understood here that this application utilizes print loss values ​​to improve the attack performance of test images generated based on target adversarial patches against real-world scenarios.

[0112] Step 322: Generate the target loss value based on the printed loss value and the feature loss value.

[0113] After obtaining the printing loss value, this application can generate a target loss value based on the printing loss value and the feature loss value. Specifically, in step 322, there are multiple ways to "generate a target loss value based on the printing loss value and the feature loss value". For example, the printing loss value and the feature loss value can be weighted to obtain a target weighted loss value; the target weighted loss value is then used as the target loss value.

[0114] In step 322, the method of "generating the target loss value based on the printing loss value and the feature loss value" can be as follows: calculate the smoothing loss value of the adversarial patch, which is obtained based on the feature distance between adjacent pixels in the adversarial patch; generate the target loss value based on the smoothing loss value, the printing loss value, and the feature loss value.

[0115] It is understood here that the smoothing loss value can represent the smoothness of the adversarial patch. This application utilizes the smoothing loss value to improve the realism of test images generated based on the target adversarial patch, thereby enhancing the robustness of the attack.

[0116] It is understood here that this application can combine smoothing loss values ​​to improve the attack performance of the target adversarial patch. Specifically, the step "calculating the smoothing loss value of the adversarial patch" can be performed as follows: the pixels in the adversarial patch are mapped to obtain the spatial position of the pixels in the adversarial patch; based on the spatial position, the feature distance between adjacent pixels in the adversarial patch is calculated; based on the feature distance, the smoothing loss value of the adversarial patch is determined. Among them, a smoothing loss function can be used to calculate the smoothing loss value of the adversarial patch based on the spatial position. The smoothing loss function can be as shown in formula (5):

[0117]

[0118] Among them, l s It can refer to the smoothing loss value; p i,j It can refer to the spatial position of the pixel in the i-th row and j-th column of the adversarial patch; p i+1,j It can refer to the spatial position of the pixel in the (i+1)th row and jth column of the adversarial patch; p i,j+1 It can refer to the spatial position of the pixel in the i-th row and j+1-th column of the adversarial patch; (p i,j -p i+1,j (p) represents the feature distance between the pixel in row i and column j and the pixel in row i+1 and column j; i,j -p i,j+1 ) represents the feature distance between the pixel in row i and column j and the pixel in row i and column j+1.

[0119] After obtaining the smoothed loss value, this application can combine the smoothed loss to improve the attack performance of the target adversarial patch. Specifically, the step of "generating the target loss value based on the smoothed loss value, the printed loss value, and the feature loss value" can be done by: obtaining the weights corresponding to the smoothed loss value, the printed loss value, and the feature loss value respectively; and fusing the smoothed loss value, the printed loss value, the feature loss value, and the weights to obtain the target loss value.

[0120] Specifically, the process of "fusing the smoothing loss value, the printing loss value, the feature loss value, and the weights to obtain the target loss value" can be found in formula (6):

[0121] l = l adv +α*l p +β*l s Formula (6)

[0122] Where l can refer to the target loss value; l adv It can refer to the feature loss value; l p It can refer to the printing loss value; α can refer to the weight corresponding to the printing loss value; l s It can refer to the smoothing loss value; β can refer to the weight corresponding to the smoothing loss value.

[0123] Step 104: If the target loss value does not meet the preset convergence condition, then based on the target loss value, update the candidate adversary patch to obtain the updated patch, and use the updated patch as the candidate adversary patch until the target loss value meets the preset convergence condition to obtain the target adversary patch.

[0124] The preset convergence condition can be set as either an iteration count threshold or a loss value threshold. For example, when the number of iterations corresponding to the target loss value reaches the iteration count threshold, the target loss value satisfies the preset convergence condition; when the number of iterations corresponding to the target loss value does not reach the iteration count threshold, the target loss value does not satisfy the preset convergence condition. As another example, when the target loss value is less than or equal to the loss value threshold, the target loss value satisfies the preset convergence condition; when the target loss value is not less than or equal to the loss value threshold, the target loss value does not satisfy the preset convergence condition.

[0125] After obtaining the target loss value, this application can update the candidate adversary patch. Specifically, the step of "updating the candidate adversary patch based on the target loss value to obtain the updated patch" can also be: using an optimizer to update the candidate adversary patch based on the target loss value to obtain the updated patch.

[0126] In this embodiment of the application, the step "update the candidate adversary patch based on the target loss value to obtain the updated patch" can also be: update the candidate adversary patch according to the target loss value to obtain the initial updated patch; trim the initial updated patch to obtain the updated patch.

[0127] The process of "updating candidate adversarial patches based on target loss values ​​to obtain initial updated patches" can be described as follows: using an optimizer to update candidate adversarial patches based on target loss values ​​to obtain initial updated patches.

[0128] After obtaining the initial update patch, this application can trim invalid pixels from the initial update patch. Specifically, the step of "trimming the initial update patch to obtain the update patch" can be done by: obtaining target pixels in the initial update patch whose pixel values ​​are not within a preset range; trimming the target pixels to obtain the update patch.

[0129] It is understandable here that, since some pixels in the initial update patch are invalid, the initial update patch needs to be trimmed to obtain a valid update patch. The update patch can be represented by formula (7):

[0130] p = clip 0,1 (p1) Formula (7)

[0131] Where p can refer to an update patch; p1 can refer to an initial update patch. Regarding formula (7), it can be understood that the preset numerical range can be greater than or equal to 0 and less than or equal to 1. This application can crop out target pixels with pixel values ​​less than 0 or greater than 1.

[0132] After obtaining the target patch, this application can test the preset image processing model based on the target patch. Specifically, this application can generate a test image based on the target patch and test the preset image processing model based on the test image.

[0133] The step "generate test image based on target patch" can be done by: obtaining the image to be processed corresponding to the preset image processing model and identifying the target position of the candidate object in the image to be processed; and overlaying the target patch with the image to be processed according to the target position to obtain the test image.

[0134] The step "obtain the image to be processed corresponding to the preset image processing model" can be done by obtaining the image to be processed corresponding to the preset image processing model from the cloud or local database.

[0135] After obtaining the image to be processed, in order to generate a test image, this application needs to identify the target location of the candidate object in the image to be processed. Specifically, the method of "identifying the target location of the candidate object in the image to be processed" can be: calling the object recognition neural network model to identify the target location of the candidate object in the image to be processed.

[0136] After obtaining the image to be processed, this application can test the preset image processing model. Specifically, the step of "testing the preset image processing model based on the test image" can be performed as follows: using the preset image processing model, object recognition is performed on the candidate objects in the test image to obtain the recognition result; based on the recognition result, the test result of the test image on the preset image processing model is determined.

[0137] In this embodiment, candidate adversarial patches can be obtained and superimposed on an initial image including the target object to obtain a candidate adversarial image. Then, feature extraction can be performed on the candidate adversarial image to obtain adversarial image features that do not include object features of the target object in the initial image, or adversarial image features that include some features of the target object in the initial image. In this way, the candidate adversarial patch can be updated using the adversarial image features and the initial image features of the initial image to obtain an excellent target adversarial patch, thereby significantly improving the versatility of the target adversarial patch and bringing a better user experience.

[0138] To facilitate understanding of the above-described embodiments, specific examples are provided below. In this example, the preset image processing model is a pedestrian detection model. Figure 3 As shown, the anti-patch processing method provided in this application embodiment is as follows: steps S201-S213:

[0139] S201. Obtain the initial image.

[0140] The initial image can be denoted as x.

[0141] S202. Obtain the weights corresponding to the smoothing loss value, the printing loss value, and the feature loss value, respectively.

[0142] S203. Extract features from the initial image to obtain the initial image features corresponding to the initial image.

[0143] The initial image features can be denoted as att. ori .

[0144] S204, Obtain candidate adversarial patches.

[0145] In this application, candidate adversarial patches can be randomly initialized to obtain initialized candidate adversarial patches, and the initialized candidate adversarial patches can be used as candidate adversarial patches.

[0146] S205. Transform the candidate adversarial patch to obtain the transformed adversarial patch.

[0147] Here, the candidate adversarial patch can be denoted as Patch p, and the transformed adversarial patch can be denoted as

[0148] S206. Overlay the transformed adversarial patch with the initial image to obtain a candidate adversarial image.

[0149] Among them, the candidate adversarial image can be denoted as

[0150] S207. Extract features from candidate adversarial images to obtain the adversarial image features corresponding to the candidate adversarial images.

[0151] Among them, adversarial image features can be denoted as:

[0152] S208. Calculate the feature loss value of the candidate adversarial image based on the adversarial image features and the initial image features.

[0153] S209. Identify adversarial patches in candidate adversarial images and calculate the printing loss value of the adversarial patches based on them.

[0154] S210. Calculate the smoothed loss value of the adversarial patch.

[0155] S211. The smoothing loss value, the printing loss value, the feature loss value, and the weights are fused to obtain the target loss value.

[0156] If the target loss value does not meet the preset convergence condition, then proceed to step S212; if the target loss value meets the preset convergence condition, then proceed to step S214.

[0157] S212. Based on the target loss value, update the candidate adversary patch to obtain the initial update patch.

[0158] Specifically, the gradient can be calculated based on the fused loss; then, the optimizer updates the candidate adversarial patch based on the gradient to obtain the updated patch. Finally, the optimizer sets the gradient to 0. The optimizer can be either the Adam optimizer or the SGD optimizer.

[0159] S213. Trim the initial update patch to obtain an update patch, and use the update patch as a candidate adversary patch.

[0160] S214. Use the update patch as the target patch.

[0161] In this application, the update of candidate adversarial patches can be performed in T-step iterations, and each iteration can be performed in steps S204-S213.

[0162] Based on the above, this application can use test images to test a preset image processing model. The test results of this application are comparable to those of existing technologies. Figure 4 and Figure 5 As shown.

[0163] against Figure 4 The preset image processing model can be a Yolov5 neural network model, among which, for Figure 4 The Yolov5 neural network model can be a white-box model. The width and height of the candidate adversarial patch are set to 100*150. In formula (6), the weight α corresponding to the printed loss value and the weight β corresponding to the smooth loss value are set to 1 and 0.1 respectively. The optimizer used is the Adam optimizer, the initial learning rate is set to 0.03, the maximum number of steps is set to 100, and the transformation parameters for the random transformation of the candidate adversarial patch can be configured according to the requirements.

[0164] Figure 4 The first row and first column represent the initial image, which is not combined with the target adversarial patch; the first row and second column represent the gradient attention map corresponding to the initial image, which is the visualization result of the initial image features. In the gradient attention map, the brighter the position, the larger the value of the gradient attention map, that is, the greater the contribution to the classification result; the first row and third column represent the detection result of the preset image processing model for pedestrians. The preset image processing model can detect pedestrians normally.

[0165] The first column of the second row represents the test image, and the second column of the second row represents the gradient attention map corresponding to the test image. The gradient attention map corresponding to the test image is mostly blue and does not contain areas with high brightness, indicating that the preset image processing model does not respond to pedestrians in the test image. The third column of the second row represents the detection results of the preset image processing model for pedestrians. It can be seen that the preset image processing model can no longer detect pedestrians in the test image, indicating that the attack was successful.

[0166] against Figure 5 When the preset image processing model is a black-box model, this application can test the preset image processing model. The preset image processing model may include the Faster-RCNN neural network model and the SSD neural network model. Existing technologies may use the YO1O neural network model.

[0167] Figure 5 In the middle, the three images in the first row correspond to respectively Figure 4 The three images in the first row. Figure 5 In the diagram, the first column of the second row represents the test image corresponding to the Faster-RCNN neural network model, the second column of the second row represents the gradient attention map corresponding to the test image corresponding to the Faster-RCNN neural network model, and the third column of the second row represents the pedestrian detection results of the Faster-RCNN neural network model.

[0168] Figure 5 In the image, the first column of the third row represents the test image corresponding to the SSD neural network model, the second column of the third row represents the gradient attention map corresponding to the test image corresponding to the SSD neural network model, and the third column of the third row represents the pedestrian detection result of the SSD neural network model.

[0169] from Figure 5 As can be seen, the highlighted areas in the gradient attention maps extracted from the test images by both the Faster-RCNN and SSD neural network models have essentially disappeared, and neither the Faster-RCNN nor the SSD neural network models can detect pedestrians in the test images. This demonstrates that the target adversarial patch in this application exhibits black-box transferability.

[0170] This application can pre-set gradient attention maps for specific categories of image processing models to achieve adversarial attacks. Furthermore, since gradient attention maps have extremely high similarity to different pre-set image processing models, the black-box transfer capability of this application is improved.

[0171] This application can improve the versatility of adversarial patches. When this application is tested on a preset image processing model, it can verify the robustness and security of the preset image processing model, such as a pedestrian detection model. Pedestrian detection models can be widely used in artificial intelligence systems, vehicle driver assistance systems, intelligent robots, intelligent video surveillance, human behavior analysis, intelligent transportation, and other fields, and have significant practical implications. For example, in intelligent driving systems, vehicles perceive their surroundings and then control the vehicle based on the perception results. Pedestrians are a crucial detection target. False or missed detections of pedestrians can lead to mis-braking or failure to brake, or even traffic accidents. Therefore, the robustness and security of pedestrian detection algorithms are extremely important.

[0172] Adversarial attacks against pedestrian detection tasks have made some progress. Most algorithms are white-box attack algorithms, which are difficult to transfer to black-box models; a few are black-box attack algorithms, but the success rate of black-box transfer is very low. The candidate adversarial patch in this application can implement both black-box and white-box attack methods, and can be used to verify the security and robustness of different pedestrian detection models.

[0173] The above describes an anti-patch processing method in the embodiments of this application. The anti-patch processing device (e.g., a server) that performs the above anti-patch processing method will be described below.

[0174] See Figure 6 ,like Figure 6 The schematic diagram of an adversarial patch processing device shown is applicable to servers in adversarial patch optimization scenarios that require improved adversarial patch versatility. This application can overlay the candidate adversarial patch with an initial image to obtain a candidate adversarial image. In this way, when extracting features from the candidate adversarial image, adversarial image features that do not contain the target object or weaken some features of the target object can be obtained. Thus, the feature loss value between the adversarial image features with similar features and the initial image features can be calculated to obtain a target loss value. Based on the target loss value, the candidate adversarial patch can be optimized to obtain a target adversarial patch with excellent and universal properties. The adversarial patch processing device in this embodiment can achieve the above-described... Figure 2 The steps of the anti-patch processing method executed in the corresponding embodiments are described below. The functions implemented by the anti-patch processing device can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The anti-patch processing device may include an input / output module 601 and a processing module 602. The functional implementation of the input / output module 601 and the processing module 602 can be found in [reference]. Figure 2The operations performed in the corresponding embodiments will not be described in detail here.

[0175] For example, input / output module 601 is configured to acquire candidate adversarial patches and overlay the candidate adversarial patches with an initial image including the target object to obtain a candidate adversarial image;

[0176] Processing module 602 is configured to extract features from candidate adversarial images to obtain adversarial image features of candidate adversarial images. The adversarial image features do not include object features of the target object in the initial image, or include some features of the target object in the initial image.

[0177] The processing module 602 is also configured to calculate the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value;

[0178] The processing module 602 is also configured to update the candidate adversary patch based on the target loss value if the target loss value does not meet the preset convergence condition, obtain the updated patch, and use the updated patch as the candidate adversary patch until the target loss value meets the preset convergence condition and the target adversary patch is obtained.

[0179] In some implementations, the processing module 602 is specifically configured to fuse the adversarial image features and the initial image features to obtain fused image features; based on the fused image features, determine the feature loss value of the candidate adversarial image, and based on the feature loss value, determine the target loss value.

[0180] In some implementations, the processing module 602 is specifically configured to identify adversarial patches in the candidate adversarial image, and calculate a printing loss value for the adversarial patch based on the adversarial patch, wherein the printing loss value characterizes the color printing loss value and / or texture printing loss value of the adversarial patch; and generate a target loss value based on the printing loss value and the feature loss value.

[0181] In some implementations, the processing module 602 is specifically configured to calculate a smoothing loss value for the adversarial patch, the smoothing loss value being obtained based on the feature distance between adjacent pixels in the adversarial patch; and to generate a target loss value based on the smoothing loss value, the printing loss value, and the feature loss value.

[0182] In some implementations, the processing module 602 is specifically configured to obtain the weights corresponding to the smoothing loss value, the printing loss value, and the feature loss value respectively; and to fuse the smoothing loss value, the printing loss value, the feature loss value, and the weights to obtain the target loss value.

[0183] In some implementations, the processing module 602 is specifically configured to update the candidate adversarial patch according to the target loss value to obtain an initial updated patch; and to trim the initial updated patch to obtain an updated patch.

[0184] In some implementations, the processing module 602 is specifically configured to acquire target pixels whose pixel values ​​are not within a preset range in the initial update patch; and to crop the target pixels to obtain the update patch.

[0185] In some implementations, the processing module 602 is specifically configured to call a preset feature extraction model to classify objects in the candidate adversarial image to obtain object categories; obtain the actual object category of the object; and calculate the adversarial image features corresponding to the candidate adversarial image based on the object category and the actual object category.

[0186] In some implementations, the processing module 602 is specifically configured to calculate the category loss value between the object category and the actual object category; and to map the category loss value to obtain the adversarial image features corresponding to the candidate adversarial image.

[0187] In this embodiment, the input / output module 601 can acquire candidate adversarial patches and overlay them with an initial image including the target object to obtain a candidate adversarial image. Then, the processing module 602 can extract features from the candidate adversarial image to obtain adversarial image features that do not include the object features of the target object in the initial image, or adversarial image features that include some features of the target object in the initial image. Thus, the processing module 602 can calculate the target loss value based on the adversarial image features and the initial image features of the initial image. The processing module 602 can then update the candidate adversarial patches based on the target loss value to obtain excellent target adversarial patches, thereby significantly improving the versatility of the target adversarial patches and bringing a better user experience.

[0188] The anti-patch processing device 60 in the embodiments of this application has been described above from the perspective of modular functional entities. The anti-patch processing device in the embodiments of this application will be described below from the perspective of hardware processing.

[0189] It should be noted that, Figure 6 The physical device corresponding to the input / output module 601 shown can be a transceiver, radio frequency circuit, communication module, and input / output (I / O) interface, etc., and the physical device corresponding to the processing module 602 can be a processor.

[0190] Figure 6 The devices shown can all have the following characteristics: Figure 7 The structure shown, when Figure 6 The anti-patch processing device 60 shown has, as Figure 7 When the structure shown is used, Figure 7The processor and transceiver in the device can perform the same or similar functions as the input / output module 601 and processing module 602 provided in the aforementioned device embodiments. Figure 8 The memory storage processor in the memory needs to call the computer program when executing the above-mentioned anti-patch processing method.

[0191] This application also provides a terminal device, such as... Figure 8 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0192] Figure 8 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 8 The mobile phone includes components such as a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will understand that... Figure 8 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0193] The following is combined Figure 8 A detailed introduction to each component of a mobile phone:

[0194] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0195] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0196] The input unit 1030 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1080, and can also receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, other input devices 1032 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0197] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1040 may include a display panel 1041, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display. Further, a touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 based on the type of touch event. Although in Figure 8 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0198] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0199] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the mobile phone. The audio circuit 1060 converts the received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts the collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and then processed by the processor 1080 before being transmitted via the RF circuit 1010 to, for example, another mobile phone, or the audio data can be output to the memory 1020 for further processing.

[0200] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 1070, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 8 The Wi-Fi module 1070 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0201] The processor 1080 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1020 and calls data stored in the memory 1020 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 1080 may include one or more processing units; optionally, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 1080.

[0202] The mobile phone also includes a power supply 1090 (such as a battery) that supplies power to various components. Optionally, the power supply can be logically connected to the processor 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0203] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0204] In this embodiment of the application, the processor 1080 included in the mobile phone also has a process for controlling the execution of the anti-patch processing method executed by the anti-patch processing device.

[0205] This application also provides a server; please refer to [link / reference]. Figure 9 , Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1122 (e.g., one or more processors) and memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 may be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.

[0206] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0207] The steps performed by the server in the above embodiments can be based on this Figure 9 The structure of server 1100 shown. For example, as in the above embodiment, by Figure 6 The steps performed by the anti-patch processing device 60 shown can be based on this Figure 9 The server architecture is shown. For example, the central processing unit 1122 performs the following operations by calling instructions from memory 1132:

[0208] Candidate adversarial patches are obtained through input / output interface 1158, and the candidate adversarial patches are superimposed on the initial image including the target object to obtain a candidate adversarial image. Features are extracted from the candidate adversarial image to obtain adversarial image features. The adversarial image features do not include the object features of the target object in the initial image, or they include some features of the target object in the initial image. The feature loss value between the adversarial image features and the initial image features of the initial image is calculated to obtain the target loss value. If the target loss value does not meet the preset convergence condition, the candidate adversarial patch is updated based on the target loss value to obtain an updated patch, and the updated patch is used as a candidate adversarial patch until the target loss value meets the preset convergence condition to obtain the target adversarial patch.

[0209] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0210] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0211] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or modules, and may be electrical, mechanical, or other forms.

[0212] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0213] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0214] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various alternative implementations of each of the above aspects.

[0215] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0216] A computer program product includes one or more computer instructions. When a computer program is loaded and executed on a computer, it produces, in whole or in part, the flow or function according to the embodiments of this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0217] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A method for handling anti-patch issues, characterized in that, The method includes: Candidate adversarial patches are obtained and superimposed on an initial image including the target object to obtain a candidate adversarial image; Feature extraction is performed on the candidate adversarial image to obtain the adversarial image features of the candidate adversarial image. The adversarial image features do not include the object features of the target object in the initial image, or they include some features of the target object in the initial image. Calculate the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value; If the target loss value does not meet the preset convergence condition, the candidate adversarial patch is updated based on the target loss value to obtain an updated patch, and the updated patch is used as the candidate adversarial patch until the target loss value meets the preset convergence condition to obtain the target adversarial patch. The step of extracting features from the candidate adversarial images to obtain the adversarial image features of the candidate adversarial images includes: A preset feature extraction model is invoked to classify the objects in the candidate adversarial image to obtain the object categories; Obtain the actual object category of the object, and calculate the category loss value between the object category and the actual object category; The category loss value is mapped to obtain the adversarial image features corresponding to the candidate adversarial image. The adversarial image features are gradient attention maps generated by mapping the candidate adversarial image to the category loss value corresponding to the candidate adversarial image. The gradient attention map is used to indicate the contribution of each pixel in the candidate adversarial image to the classification result output by the preset feature extraction model.

2. The anti-patch processing method according to claim 1, characterized in that, The step of calculating the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value includes: The adversarial image features and the initial image features are fused to obtain fused image features; Based on the fused image features, the feature loss value of the candidate adversarial image is determined, and based on the feature loss value, the target loss value is determined.

3. The anti-patch processing method according to claim 2, characterized in that, Determining the target loss value based on the feature loss value includes: An adversarial patch is identified in the candidate adversarial image, and a printing loss value for the adversarial patch is calculated based on the adversarial patch. The printing loss value represents the color printing loss value and / or texture printing loss value of the adversarial patch. The target loss value is generated based on the printed loss value and the feature loss value.

4. The anti-patch processing method according to claim 3, characterized in that, The step of generating the target loss value based on the printed loss value and the feature loss value includes: Calculate the smoothing loss value of the adversarial patch, which is obtained based on the feature distance between adjacent pixels in the adversarial patch; The target loss value is generated based on the smoothing loss value, the printing loss value, and the feature loss value.

5. The anti-patch processing method according to claim 1, characterized in that, The step of updating the candidate adversarial patch based on the target loss value to obtain the updated patch includes: Based on the target loss value, update the candidate adversarial patch to obtain the initial updated patch; The initial update patch is trimmed to obtain the new update patch.

6. A device for handling anti-patch issues, characterized in that, The device includes: The input / output module is configured to acquire candidate adversarial patches and overlay the candidate adversarial patches with an initial image including the target object to obtain a candidate adversarial image; The processing module is configured to extract features from the candidate adversarial image to obtain adversarial image features of the candidate adversarial image, wherein the adversarial image features do not include object features of the target object in the initial image, or include partial features of the target object in the initial image; The processing module is also configured to calculate the feature loss value between the adversarial image features and the initial image features of the initial image to obtain the target loss value; The processing module is also configured to update the candidate adversarial patch based on the target loss value if the target loss value does not meet the preset convergence condition, to obtain an updated patch, and to use the updated patch as the candidate adversarial patch until the target loss value meets the preset convergence condition and the target adversarial patch is obtained. The processing module is further configured as follows: A preset feature extraction model is invoked to classify the objects in the candidate adversarial image to obtain the object categories; Obtain the actual object category of the object, and calculate the category loss value between the object category and the actual object category; The category loss value is mapped to obtain the adversarial image features corresponding to the candidate adversarial image. The adversarial image features are gradient attention maps generated by mapping the candidate adversarial image to the category loss value corresponding to the candidate adversarial image. The gradient attention map is used to indicate the contribution of each pixel in the candidate adversarial image to the classification result output by the preset feature extraction model.

7. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It includes instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1-5.

Images