A face picture processing method, related equipment and storage medium

Abstract

Embodiments of the present application relate to the technical field of search and provide a face picture processing method, related equipment and a storage medium. The method comprises the following steps: obtaining a target face picture of a face adversarial sample to be generated; performing regional division on the target face picture to obtain at least one first region and at least one second region, wherein the first region is a non-face key feature region, and the second region is a face key feature region; adding a first disturbance to the at least one first region and a second disturbance to the at least one second region to generate a face adversarial sample; and publishing or updating specific event information, wherein the specific event information comprises at least one face adversarial sample. The method can reduce the influence of adversarial disturbance on the visual effect of the face and improve the processing efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method for processing facial images, related equipment, and storage medium. Background Technology

[0002] In face adversarial training, face adversarial examples are first generated. Usually, the same perturbation is added to the entire face of the face image. Whether it is a global perturbation or a local perturbation, it only considers the attack effect of the face adversarial examples, but does not consider whether adding perturbations to different regions of the face image will affect the visual effect of the face.

[0003] For example, adding perturbations to non-key facial feature areas in a face image can result in very noticeable perturbations, such as colored stripes, which affect the visual effect of the face. Summary of the Invention

[0004] This application provides a face image processing method, related equipment, and storage medium, which can reduce the impact of adversarial disturbances on the visual effect of faces and improve processing efficiency.

[0005] In a first aspect, embodiments of this application provide a face image processing method from the perspective of a business server, the method comprising:

[0006] Obtain the target face image from which the adversarial face example to be generated;

[0007] The target face image is divided into regions to obtain at least one first region and at least one second region, wherein the first region is a non-facial key feature region and the second region is a facial key feature region.

[0008] At least one first region is subjected to a first perturbation, and at least one second region is subjected to a second perturbation to generate adversarial face examples; the first perturbation and the second perturbation are perturbations with different norms.

[0009] Publish or update specific event information, which includes at least one of the face adversarial samples.

[0010] In some implementations, the target face image refers to at least one image to be published by a target user that includes at least one face, or at least one image already published by a target user that includes at least one face.

[0011] Secondly, embodiments of this application provide a method for obtaining facial images from a search engine perspective, the method comprising:

[0012] Receive a search instruction, which includes key information about the target user;

[0013] In response to the search command, a set of candidate images matching the key information is obtained from the business server; wherein, the set of candidate images includes at least one adversarial face sample, which is generated based on adding a first perturbation to at least one first region and adding a second perturbation to at least one second region, wherein the first region and the second region are obtained by dividing the target face image corresponding to the target user into regions; the first region is a non-key facial feature region, and the second region is a key facial feature region;

[0014] The target image set is determined from the candidate image set as the search results for the target user, and the target image set does not include adversarial face samples.

[0015] In some implementations, generating adversarial face examples by applying a first perturbation to at least one first region and a second perturbation to at least one second region includes:

[0016] Multiple face images are classified to obtain multiple clusters;

[0017] The first perturbation is applied to at least one of the first regions of each face image in each cluster, and the second perturbation is applied to the second region to obtain the face adversarial sample.

[0018] In some implementations, the target image set refers to images that have been posted by at least one target user and include at least one human face.

[0019] Thirdly, embodiments of this application also provide a business server for implementing the face image processing method described in the first aspect above, the business server comprising:

[0020] The input / output module is used to acquire the target face image to be generated as an adversarial example;

[0021] The processing module is used to divide the target face image obtained by the input and output into regions to obtain at least one first region and at least one second region, wherein the first region is a non-facial key feature region and the second region is a facial key feature region.

[0022] At least one first region is subjected to a first perturbation, and at least one second region is subjected to a second perturbation to generate adversarial face examples; the first perturbation and the second perturbation are perturbations with different norms.

[0023] The input / output module is also used to publish or update specific event information, which includes at least one adversarial face sample obtained by the processing module.

[0024] In some implementations, there are multiple target face images. After the input / output module obtains the target face images to be used for generating adversarial examples, and before generating the adversarial examples, the processing module is further configured to:

[0025] Multiple face images are divided into multiple first clusters;

[0026] Multiple first-type annotation regions and multiple second-type annotation regions are displayed on the face images in each first cluster. The first-type annotation regions are used to locate the regions on each face image to which the first perturbation is to be added, and the second-type annotation regions are used to locate the regions on each face image to which the second perturbation is to be added.

[0027] A first perturbation is added in batches to the first type of labeled regions of the face images in each first cluster, and a second perturbation is added in batches to the second type of labeled regions of the face images in each cluster, so as to obtain multiple face adversarial samples.

[0028] In some implementations, the processing module is specifically used for:

[0029] Determine the feature information of the target features in each first region;

[0030] Obtain the first perturbation that matches the feature information of the target features in each first region;

[0031] The first perturbation is added to the region to which the corresponding target feature belongs.

[0032] In some implementations, the processing module is specifically used for:

[0033] A first perturbation is applied to at least one of the first regions;

[0034] The multiple face images with the first perturbation added are classified to obtain multiple second clusters;

[0035] The second perturbation is applied to at least one of the second regions of each face image in each second cluster to obtain the face adversarial sample.

[0036] In some implementations, the processing module is specifically used for:

[0037] Multiple face images are classified to obtain multiple clusters;

[0038] The first perturbation is applied to at least one of the first regions of each face image in each cluster, and the second perturbation is applied to the second region to obtain the face adversarial sample.

[0039] In some embodiments, the processing module is further configured to:

[0040] The input / output module receives a first message from the user terminal. The first message includes specific event information to be published by the target user, and the specific event information includes at least one face image.

[0041] The face images that meet the preset privacy protection conditions in the specific event information are set as the target face images.

[0042] In some implementations, the processing module is specifically used for:

[0043] Identify target users who meet privacy protection requirements;

[0044] The input / output module obtains the historical image set published by the target user within a historical time period, and the historical image set includes at least one unprocessed candidate historical image of the target user.

[0045] The candidate historical images that meet the preset privacy protection conditions are set as the target face images.

[0046] In some embodiments, the processing module is further configured to:

[0047] The input / output module receives a second message from the user terminal, the second message indicating that at least one perturbation should be added to at least one face image in the specific event information to be published by the target user;

[0048] According to the instructions of the second message, the face image that meets the preset privacy protection conditions in the specific event information is set as the target face image.

[0049] Fourthly, embodiments of this application also provide a search engine for implementing the face image acquisition method described in the second aspect above, the search engine comprising:

[0050] The transceiver module is used to receive search instructions, which include key information about the target user.

[0051] The processing module is configured to respond to the search command received by the transceiver module, and obtain a set of candidate images matching the key information from the business server through the transceiver module; wherein, the set of candidate images includes at least one adversarial face sample, the adversarial face sample is generated based on adding a first perturbation to at least one first region and adding a second perturbation to at least one second region, the first region and the second region are obtained by dividing the target face image corresponding to the target user into regions; the first region is a non-key facial feature region, and the second region is a key facial feature region; the first perturbation and the second perturbation are perturbations with different norms;

[0052] The processing module is further configured to determine a target image set from the candidate image set as the search results for the target user, wherein the target image set does not include adversarial face samples.

[0053] Fifthly, embodiments of this application also provide a processing device, including a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, it executes the steps in the first aspect, any implementation of the first aspect, or any face image processing method provided in the second aspect of this application.

[0054] Sixthly, embodiments of this application also provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the steps in the first aspect, any implementation thereof, or any face image processing method provided in the second aspect of this application.

[0055] Compared with the prior art, in this embodiment, the feature points in the face image are first divided into regions according to non-facial key features and facial key features to obtain at least one first region and at least one second region. Since the first region is a non-facial key feature region and the second region is a facial key feature region, when generating adversarial examples of the face later, a first perturbation and a second perturbation are added according to the features contained in the first region and the second region respectively. Since the first perturbation and the second perturbation are perturbations with different norms and large differences, when the first perturbation is added to the first region, the appearance of the first region after adding the first perturbation is more in line with the original features in the first region compared with the uniform addition of perturbation in the prior art. Similarly, when the second perturbation is added to the second region, the appearance of the second region after adding the second perturbation is more in line with the original features in the second region compared with the uniform addition of perturbation in the prior art. In short, by adding different perturbations to different regions of the target face image, the final generated adversarial sample can ensure the attack effect without affecting the appearance of the original facial features on the target face image. This can reduce viewer discomfort or make the sample more subtle and less noticeable, thereby improving the user experience. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1This is a schematic diagram of an application environment of the communication system in an embodiment of this application;

[0058] Figure 2 This is a flowchart illustrating a face image processing method in an embodiment of this application;

[0059] Figure 3a This is a schematic diagram of the segmentation of a target face image in an embodiment of this application;

[0060] Figure 3b This is a schematic diagram illustrating the addition of different perturbations to the segmented target face image in an embodiment of this application;

[0061] Figure 4 This is another flowchart illustrating the face image processing method in the embodiments of this application;

[0062] Figure 5a This is a schematic diagram illustrating the addition of perturbation to the facial image of the target user to be posted on Weibo in an embodiment of this application;

[0063] Figure 5b This is a schematic diagram illustrating the addition of perturbation to a complete face image synthesized from a nine-grid image uploaded to a social media platform, as described in this application embodiment.

[0064] Figure 5c This is a schematic diagram illustrating the addition of perturbation to the facial images of target users historically posted on social media platforms, as described in this application embodiment.

[0065] Figure 5d This is a schematic diagram illustrating the addition of perturbation to facial images of target users and specific users historically posted on social media platforms, as described in this application embodiment.

[0066] Figure 5e This is a schematic diagram illustrating the batch addition of different perturbations to segmented target face images in an embodiment of this application;

[0067] Figure 6a This is a flowchart illustrating a method for obtaining face images in an embodiment of this application;

[0068] Figure 6b This is a flowchart illustrating a method for obtaining face images in an embodiment of this application;

[0069] Figure 6c This is a schematic diagram illustrating multi-path parallel processing of the image set to be processed in an embodiment of this application;

[0070] Figure 7 This is a schematic diagram of the structure of an adversarial sample generation device in an embodiment of this application;

[0071] Figure 8 This is a schematic diagram of the structure of a search device in an embodiment of this application;

[0072] Figure 9 This is a schematic diagram of the structure of a physical device implementing the face image processing method in the embodiments of this application;

[0073] Figure 10 This is a schematic diagram of the structure of a mobile phone implementing the face image processing method in an embodiment of this application;

[0074] Figure 11 This is a schematic diagram of the structure of a server implementing the face image processing method in an embodiment of this application. Detailed Implementation

[0075] The terms "target," "candidate," "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects with the same attributes and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division; in actual applications, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not performed. Additionally, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, and the indirect coupling or communication connection between modules may be electrical or other similar forms; none of these are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0076] This application provides a face image processing method, apparatus, and storage medium, which can be used to perform adversarial perturbation processing on images to be published and previously published, thereby generating face adversarial examples. This makes it impossible for a search engine (i.e., a search device) based on a neural network model to directly identify the target face image matching the specific user even after crawling face images of a specific user from the network (e.g., crawling face images previously published by the target user from a business server) and preprocessing the face images.

[0077] In some implementations, this solution is applied to, for example... Figure 1 The communication system shown may include at least one service server and at least one search device. Figure 1 Only one search device (a) is shown in the diagram. This search device can be a server or a terminal. When it is a terminal, it has a client installed. The clients installed on each search device can be the same or different versions of the client, or clients with the same or different operating systems. This application does not limit the version of the client or the type of operating system corresponding to the client. A detailed description follows.

[0078] Business server: This server can be used to provide image search services to various search devices, as well as to provide users with image upload, download, or preprocessing services. For example, the server can perform adversarial perturbation processing on images uploaded by users and then update them online. A face adversarial example generation device can be deployed on this business server.

[0079] Search device: It can crawl facial images or videos that match the key information of the target user from the business server, then preprocess the images or videos, and then identify the preprocessed images to search for images that include the face of the target user.

[0080] When based on such Figure 1 When the communication system described above implements the face image processing method and face image acquisition method, the service server can perform adversarial perturbation processing on the face image 'a' uploaded by the target user (e.g., adding small perturbations to the cheeks and forehead, and larger perturbations in other locations), and then update and publish it online. On the one hand, this prevents the search device from recognizing that face image 'a' belongs to the target user after crawling it from the service server, thus interfering with the face recognition function of the search device. On the other hand, because the service server adds small perturbations to the cheeks and forehead and larger perturbations in other locations during the face adversarial example generation process, it can effectively reduce the impact of adversarial perturbations on the visual effect of the face.

[0081] The solutions provided in this application involve technologies such as Artificial Intelligence (AI), Natural Language Processing (NLP), and Machine Learning (ML), and are specifically illustrated through the following embodiments:

[0082] AI, or Artificial Intelligence, refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, Artificial Intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a manner similar to human intelligence. Artificial Intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0083] AI technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0084] NLP is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. Natural Language Processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close connection with linguistic research. Natural Language Processing techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0085] Targeting the facial recognition field in artificial intelligence, this application can generate a general perturbation with high attack robustness using a set of facial images. By adding the general perturbation to the facial images in the set, the image classification model cannot accurately identify the perturbated facial images, thereby achieving an attack on the image classification model.

[0086] It should be specifically noted that the servers involved in this application's embodiments (e.g., business servers, search engines) can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The adversarial example generation device involved in this application's embodiments can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, personal digital assistant, etc., but is not limited to these. The adversarial example generation device and the server can be directly or indirectly connected via wired or wireless communication, and this application's embodiments do not impose any restrictions.

[0087] When a user's image is disseminated on public social networks, the user does not want third parties to be able to use web crawlers, search engines, or other means to search for their own facial image using AI image recognition technology, such as facial recognition. Therefore, they need to add adversarial perturbations to the image to interfere with facial recognition. Simultaneously, the user wants their image to be viewed in their social circle without affecting viewers' judgment of their appearance; therefore, the user does not want the adversarial perturbations to excessively affect the visual effect of their appearance. To address this, the embodiments of this application mainly adopt the following technical solutions:

[0088] 1. Images to be published and previously published are partitioned and subjected to different perturbations, and corresponding adversarial perturbation processing is performed to generate adversarial examples of faces. This makes it impossible for a search engine based on a neural network model to directly identify the target face image matching the specific user even after crawling face images of a specific user from the web (e.g., crawling face images previously published by the target user from the business server) and preprocessing the face images.

[0089] 2. By applying different perturbations to different regions, i.e., applying the first perturbation in batches to the first region and the second perturbation in batches to the second region, the processing efficiency can be effectively improved.

[0090] The following describes the face image processing method provided in the embodiments of this application.

[0091] See Figure 2 , Figure 2 This document illustrates a flowchart of a face image processing method according to an embodiment of this application. The following describes a face image processing method provided by this embodiment, which is executed by a face adversarial example generation device, a business server, or a business server deploying a face adversarial example generation device. This embodiment uses a business server as the execution entity, which is equivalent to the face adversarial example generation device and is not distinguished from it. This embodiment includes:

[0092] 101. Obtain the target face image to be generated as an adversarial example.

[0093] In this context, adversarial examples of faces refer to facial images with added perturbation patterns. These perturbation patterns can be watermarks, text, etc., and can be regular or irregular in shape, without limitation. Each perturbation pattern corresponds to a unique label for the adversarial example, preventing search engines from identifying the user identity associated with the perturbation pattern-added adversarial example. This can lead to misidentification, underidentification, or failure to identify the user by search engines.

[0094] In some implementations, the target face image includes either a or b:

[0095] a. At least one set of images to be published by the target user, including at least one face (for ease of reference, this will be referred to as the set of images to be published, and will not be elaborated further).

[0096] For example, on March 29, 2022, at 18:20:20, after Zhang San edited the text of his "mood" and uploaded several corresponding photos of himself and / or his friends on a social media platform, before Zhang San clicked the "post" button, the business server could proactively prompt Zhang San whether he wanted to add perturbation patterns to "Zhang San" and / or "Zhang San's friends"; or, Zhang San could proactively trigger the operation of adding perturbation patterns to "Zhang San" and / or "Zhang San's friends"; or, the business server could automatically add perturbation patterns to "Zhang San" and / or "Zhang San's friends" in the photos Zhang San was about to post based on Zhang San's historical behavior data (such as data on adding perturbation patterns).

[0097] b. At least one image posted by the target user that includes at least one face (for ease of reference, it will be referred to as the historical image set, and will not be elaborated further).

[0098] For example, after Zhang San edited his "mood" text and uploaded multiple historical photos, including himself and / or his friends, on a social media platform between January 1, 2022 and March 29, 2022, the service server can proactively prompt Zhang San whether he needs to add perturbation patterns to "Zhang San" and / or "Zhang San's friends"; alternatively, Zhang San can proactively trigger the operation of adding perturbation patterns to "Zhang San" and / or "Zhang San's friends"; or, the service server can automatically add perturbation patterns to "Zhang San" and / or "Zhang San's friends" in historical photos posted by Zhang San between January 1, 2022 and March 29, 2022, based on Zhang San's historical behavior data (e.g., data on adding perturbation patterns). Optionally, the service server can proactively add perturbation patterns to the historical photo set based on Zhang San's historical behavior data without limiting the execution cycle, and can set it statically or dynamically.

[0099] For the same target user, their pending image sets or previously published image sets on social media platforms can all serve as sources of the target facial images, depending on the target user's needs. This application embodiment does not limit this. These needs can be initiated or subscribed to by the target user. In some implementations, these needs can also be triggered when the business server detects a target facial image in specific information corresponding to the target user (e.g., from a pending image set or a previously published image set) that has not yet generated adversarial facial samples, prompting the target user whether they need to trigger the process of generating adversarial facial samples.

[0100] In other implementations, such as Figure 5cThe attached Weibo post consists of a nine-grid image. Two of the images in the nine-grid image are partial facial images of the target user. When the business server performs privacy protection processing on the nine-grid Weibo post containing the target user, it first locates the partial facial images containing the target user (or other specific users) in the nine-grid Weibo post. Then, it inputs these partial facial images into the target model to obtain adversarial facial samples, and then replaces the nine-grid Weibo post (i.e., publishes or updates it).

[0101] It should be noted that, for the same target user, any image containing a face in their pending image set or previously published image set on a social media platform can be considered the aforementioned target face image. This application embodiment does not limit the user identity or the number of faces corresponding to the faces included in the same image. The subsequent embodiments of this application only describe the operation of generating adversarial examples from a face region in an image. For operations involving generating adversarial examples from at least one face region of different or the same user identity in one or at least two images simultaneously or sequentially, refer to the embodiment of generating adversarial examples from a single face region in an image, which will not be repeated here.

[0102] Based on the above explanation of the sources of target face images, the following describes several methods and timing for obtaining target face images to be used to generate adversarial examples:

[0103] (1) Processing scenario where the business server publishes information to a single user in real time based on event triggering.

[0104] Specifically, the user terminal sends a first message to the service server. The first message includes specific event information to be published by the target user, and the specific event information includes at least one face image.

[0105] The service server receives the first message from the user terminal, and then sets the face image that meets the preset privacy protection conditions in the specific event information as the target face image.

[0106] In some implementations, the preset privacy protection conditions may include at least one of the following:

[0107] Including the target user's face;

[0108] Faces of specific users outside the target users;

[0109] Alternatively, it could include the face of the target user, or the face of a specific user other than the target user.

[0110] Among them, a specific user can be a user designated by the target user, or a user who has a relationship with the target user (such as an online / offline social friend relationship). This application embodiment does not limit this.

[0111] For example, the specific event information to be published could be a mood written by user A on a social media platform, such as a Weibo post or a public account article. This mood could include not only text but also photos of user A's life uploaded by that user, including user A's face and also user B's face. User B could be anyone other than user A, such as user A's friends, family, or public figures; this application embodiment does not limit this.

[0112] As can be seen, the business server automatically identifies target facial images that meet preset privacy protection conditions from the specific time information to be published by the target user. Since these preset privacy protection conditions are pre-set filtering conditions, the target facial images can be automatically filtered out. This allows the business server to quickly and specifically generate adversarial facial samples for the target user without requiring the target user to actively trigger the process of adding perturbation patterns. Therefore, the privacy and security of the target user can be guaranteed without their awareness.

[0113] (2) The business server proactively processes the historical published information of individual users in batches on a regular basis.

[0114] In some implementations, the following steps may be included:

[0115] The business server identifies target users who meet the preset privacy protection conditions;

[0116] The business server obtains the historical image set published by the target user within a historical time period, and the historical image set includes at least one unprocessed candidate historical image of the target user.

[0117] The service server sets the candidate historical images that meet the preset privacy protection conditions as the target face images.

[0118] As can be seen, the business server automatically identifies candidate historical images that meet the preset privacy protection conditions from the set of historical images posted by the target user. Since the preset privacy protection conditions are pre-set filtering conditions, on the one hand, it can automatically filter out target face images, so that the business server can quickly and specifically generate adversarial face samples for the target user without the target user actively triggering the process of adding perturbation patterns. Therefore, it can protect the privacy and security of the target user without the target user's awareness.

[0119] On the other hand, even if the target user does not actively request the business server to add a perturbation pattern to protect privacy when posting any image from the aforementioned historical image set; or even if the business server does not automatically add a perturbation pattern to protect the target user's privacy when posting any image from the aforementioned historical image set; or even if the business server does not periodically or irregularly add a perturbation pattern to protect the target user's privacy for a period of time after the target user posts any image from the aforementioned historical image set. With this solution, perturbation patterns can be added to candidate historical images in the historical image set that meet the preset privacy protection conditions in the later stages. This means that by identifying and addressing omissions or providing compensation, the target user's need for privacy and security can be improved.

[0120] (3) Scenarios where the business server processes real-time / historical information published by a single user based on user-initiated requests.

[0121] In some implementations, the following steps may be included:

[0122] The user terminal sends a second message to the business server, the second message instructing that at least one perturbation be added to at least one face image in the specific event information to be published by the target user;

[0123] After receiving the second message from the user terminal, the service server sets the face image that meets the preset privacy protection conditions in the specific event information as the target face image according to the instructions of the second message.

[0124] It is evident that by having the target user proactively request a perturbation pattern from the business server when posting any image from the aforementioned historical image set, the privacy and security of the target user can be protected.

[0125] The above three methods for obtaining the target face image of the face to be generated adversarial example can be flexibly switched, configured, or combined, and any one or at least two of them can be implemented. This application embodiment does not limit this.

[0126] 102. Divide the target face image into regions to obtain at least one first region and at least one second region.

[0127] The first region is a non-facial key feature region, such as the forehead, chin, and cheeks.

[0128] The second region is the key facial feature region, such as the facial features such as eyebrows, eyes, nose, mouth, and ears.

[0129] For example Figure 3a As shown, the dashed rectangle represents the first region, and the solid rectangle represents the second region. From... Figure 3aAs can be seen, there is one first area for the forehead, one first area for the chin, and two first areas for the cheeks. There is one second area for the eyes and one second area for the nose.

[0130] This application does not limit the shape, granularity, method, number, or area of ​​the first and second regions. The face region contains multiple non-facial key feature regions and facial key feature regions. In this application embodiment, the region can be divided according to actual needs (e.g., business requirements, accuracy, etc.). In some implementations, only at least one first region may be divided, i.e., a region specifically used to add the first perturbation; this is not limited.

[0131] 103. Add a first perturbation to at least one first region and a second perturbation to at least one second region to generate adversarial face examples.

[0132] The first perturbation and the second perturbation are perturbations with different norms. For example, the perturbation added by the first perturbation is no less than the minimum perturbation required for a successful white-box undirected attack, and no greater than the minimum perturbation required for a successful black-box migration undirected attack. For example, the first perturbation is L. ∞ Small perturbations ≤4, such as generating L using momentum-based adversarial example generation methods. ∞ Perturbations ≤4.

[0133] The second perturbation adds a perturbation whose magnitude is no less than the minimum perturbation required for a successful undirected black-box migration attack, and no greater than n times (e.g., three times) the minimum perturbation required for a successful undirected black-box migration attack. The three-fold increase is to improve the stability and success rate of the attack. For example, the second perturbation is L. ∞ Large perturbations ≤14, such as those generated using momentum-based adversarial example generation methods. ∞ Perturbations ≤8. L ∞ ≤p represents the L of the disturbance. ∞ The norm is less than p.

[0134] by Figure 3a For example, by adding a first perturbation to the first region (e.g., adding a five-pointed star pattern inside the dashed rectangle) and a second perturbation to the second region (e.g., adding a solid dot pattern inside the solid rectangle), the adversarial face sample shown in Figure 3b is obtained.

[0135] 104. Publish or update information about specific events.

[0136] The specific event information includes at least one of the face adversarial samples.

[0137] In this embodiment of the application, publishing or updating specific event information can refer to posting some moods on a social media platform. For example, a target user posts a Weibo post (a) on the Weibo platform. This Weibo post (a) includes a face image (a) of the target user and / or a specific user. This face image (a) is an adversarial sample obtained after privacy protection processing of the face image (a') initially uploaded by user a. That is, the face image (a') is partitioned and perturbed, and the resulting adversarial sample is published as the aforementioned face image (a), i.e., face image (a) is replaced by face image (a'). For example... Figure 5a The diagram shown is a schematic diagram.

[0138] Updating the target face image with the adversarial face sample can refer to the following two scenarios (1) and (2):

[0139] (1) After processing the face images that the target user has posted on social media platforms in the past for privacy protection, they are replaced with adversarial face samples.

[0140] For example, user A uploads a Weibo post (b) on the Weibo platform. This post (b) includes a facial image (b) of a target user and / or a specific user. This facial image (b) is an adversarial sample obtained by processing the user's previously uploaded facial images (b') for privacy protection. In other words, the facial image (b') is input into a target model, and the adversarial sample output by the target model is published as the aforementioned facial image (b), i.e., facial image (b) is replaced by facial image (b'). A schematic diagram can be referenced. Figure 5c , Figure 5d I will not go into details.

[0141] In some implementations, such as Figure 5b The attached Weibo post shown is composed of a nine-grid image. Two of the images in the nine-grid image are partial facial images of the target user. When the business server performs privacy protection processing on the nine-grid Weibo post containing the target user, it first needs to locate the partial facial images containing the target user (or a specific user) from the nine-grid Weibo post. Then, it inputs these partial facial images into the target model to obtain adversarial facial samples, and then replaces the nine-grid Weibo post.

[0142] (2) After processing the face images uploaded by the target user to the financial platform in the past for privacy protection, they are replaced with face adversarial samples.

[0143] For example, user A uploads an ID card image c on a financial platform. This ID card image c includes ID card images c of target users and / or specific users. This ID card image c is a face adversarial sample obtained after privacy protection processing of the user's previously uploaded ID card image c'. That is, the ID card image c' is input into the target model, and the face adversarial sample output by the target model is uploaded as the aforementioned ID card image c, i.e., replacing ID card image c with ID card image c'. See details for further information. Figures 5a-5e The example scenarios shown are not elaborated upon.

[0144] In this embodiment, the feature points in the face image are first divided into regions according to non-facial key features and facial key features to obtain at least one first region and at least one second region. Since the first region is a non-facial key feature region and the second region is a facial key feature region, when generating adversarial examples of the face later, a first perturbation and a second perturbation are added according to the features contained in the first region and the second region respectively. Since the first perturbation and the second perturbation are perturbations with different norms and large differences, when the first perturbation is added to the first region, the appearance of the first region after adding the first perturbation is more in line with the original features in the first region compared with the uniform addition of perturbation in the prior art. Similarly, when the second perturbation is added to the second region, the appearance of the second region after adding the second perturbation is more in line with the original features in the second region compared with the uniform addition of perturbation in the prior art. In short, by adding different perturbations to different regions of the target face image, the final generated adversarial sample can ensure the attack effect without affecting the appearance of the original facial features on the target face image. This can reduce viewer discomfort or make the sample more subtle and less noticeable, thereby improving the user experience.

[0145] In some embodiments, considering the efficiency of generating adversarial examples for faces, especially when processing multiple face images in parallel, to improve processing efficiency, the target image set can be clustered before adding perturbations in batches. Specifically, the target face images are multiple, such as... Figure 4 As shown, after the business server obtains the target face image to be used for generating adversarial face examples, but before generating the adversarial face examples, the method further includes:

[0146] 201. The business server divides multiple face images into multiple first clusters.

[0147] In some implementations, the division can be based on at least one dimension, such as facial features, facial size, or region.

[0148] 202. The business server adds perturbation to each cluster in a unified manner.

[0149] Each cluster is subjected to the same perturbation in the same labeled area, which can be a fixed bounding box.

[0150] The disturbance may be applied to the first region or the second region, or may include disturbances applied to both the first region and the second region simultaneously, without limitation.

[0151] Specifically, multiple first-type annotation regions and multiple second-type annotation regions are displayed on the face images in each first cluster. The first-type annotation regions are used to locate the regions on each face image to which the first perturbation will be added, and the second-type annotation regions are used to locate the regions on each face image to which the second perturbation will be added. For example... Figure 5e As shown.

[0152] 203. The business server generates adversarial examples.

[0153] Specifically, a first perturbation is added in batches to the first type of labeled regions of the face images in each first cluster, and a second perturbation is added in batches to the second type of labeled regions of the face images in each cluster, so as to obtain multiple face adversarial samples.

[0154] It is evident that by classifying and dividing the data into clusters, applying different perturbations to each region—that is, applying the first perturbation to the first region in batches and the second perturbation to the second region in batches—processing efficiency can be effectively improved.

[0155] In some embodiments, a first perturbation can be uniformly added to a first region first, then the multiple face images with the first perturbation added can be classified to obtain multiple clusters, and then a second perturbation can be added to the second region for each cluster. Specifically, the service server adds a first perturbation to at least one first region and a second perturbation to at least one second region to generate face adversarial examples, including:

[0156] A first perturbation is applied to at least one of the first regions;

[0157] The multiple face images with the first perturbation added are classified to obtain multiple second clusters;

[0158] The second perturbation is applied to at least one of the second regions of each face image in each second cluster to obtain the face adversarial sample.

[0159] It is evident that by classifying and dividing the data into clusters, applying different perturbations to each region—that is, applying the first perturbation in batches to the first region and the second perturbation in batches to the second region—the efficiency of privacy protection processing can be effectively improved.

[0160] See Figure 6a , Figure 6a This paper illustrates a flowchart of a face image acquisition method according to an embodiment of this application. The following describes a face image acquisition method provided by this application embodiment, which is applied to a search engine. This application embodiment includes:

[0161] 601. The search engine receives the search command.

[0162] The search instructions include key information about the target user, such as the target user's name and nickname. The target user can be at least one user to be searched, and the search engine can simultaneously search for images related to at least two target users; however, this embodiment does not limit this.

[0163] 602. In response to the search command, the search engine sends an access request to the business server.

[0164] This access request is used to request a facial image that matches key information from the business server. For example, if the search instruction includes the name Zhang Wenwen, the access request could be to request a facial image tagged "Zhang Wenwen" from platforms such as Weibo and various public image libraries.

[0165] 603. The search engine obtains a set of candidate images that match the key information from the business server.

[0166] The candidate image set includes at least one adversarial face sample, and may also include at least one target face image (i.e., a face image that has not undergone privacy protection processing, or an adversarial face sample that has not undergone privacy protection processing using the technical solution of this application embodiment; this application does not limit this). The adversarial face sample is generated based on adding a first perturbation to at least one first region and adding a second perturbation to at least one second region. The first region and the second region are obtained by dividing the target face image corresponding to the target user into regions. The first region is a non-key facial feature region, and the second region is a key facial feature region.

[0167] 604. The search engine determines the target image set from the candidate image set as the search results for the target user.

[0168] The target image set does not include adversarial examples of faces. The target image set refers to images containing at least one face that have been posted by at least one target user. This target image set can originate from images containing the target user's face posted by user A, or it can originate from images containing the target user's face posted by user B; this embodiment of the application does not limit this. For details, please refer to... Figure 6b The diagram shown is just one example and will not be elaborated upon further.

[0169] Specifically, the candidate image set can be input into the face recognition model deployed in the search engine to obtain the first similarity between the target adversarial sample and the preset face image, and the second similarity between each target face image and the preset face image; the first similarity is lower than the first threshold, the second similarity is not less than the third threshold, and the first threshold is less than the third threshold;

[0170] The target image set is selected from the candidate image set, and the similarity between the target image and the preset face image is not less than the third threshold.

[0171] As can be seen, in this embodiment, because the business server adds perturbation processing to at least one image containing a face posted by the user, at least one image containing a face generates an adversarial sample. Therefore, when other search tools (such as the search engine in this embodiment) search and crawl these candidate image sets from the business server according to search instructions including the aforementioned key information, even if the search engine crawls an adversarial sample containing the target user, it cannot intelligently identify the adversarial sample matching the key information of the target user from machine vision when preprocessing (e.g., filtering) the candidate image set. That is, the target image set identified by the search engine from the candidate image set will not include the aforementioned adversarial sample. Thus, on the one hand, this embodiment can interfere with face recognition by adding adversarial perturbation, thereby resisting the search engine based on the neural network model from filtering out face images containing the target user (i.e., the aforementioned adversarial sample). On the other hand, it also effectively protects the privacy and security of user data.

[0172] Optionally, in other embodiments of this application, in addition to partitioning the target face image to be processed into parts or all of it and adding different perturbations, this solution can also generate adversarial examples of faces based on a target model trained through unsupervised learning for the part of the target face image to be processed. This ensures the adversarial effect while improving the efficiency of generating adversarial examples. These two methods can be implemented individually or in parallel, and this application does not limit this approach.

[0173] For example Figure 6c As shown, the image set to be processed includes n face images. On the business server side, the image set is first divided into a first image set and a second image set. The first image set is input into face recognition model a, and the second image set is input into face recognition model b (i.e., the target model obtained by unsupervised learning training, for example, it can be based on an unsupervised generative adversarial network). The adversarial examples output by face recognition model a are obtained by partitioning each target face image in the first image set and adding different perturbations. The adversarial examples output by face recognition model b are obtained by inputting each target face image in the second image set into face recognition model b and adding adversarial perturbations (e.g., random noise).

[0174] Figures 1 to 6b Any technical feature mentioned in the embodiments corresponding to any one of the above also applies to the embodiments of this application. Figures 7 to 11 The corresponding implementation examples will not be repeated hereafter.

[0175] The above describes a face image processing method in the embodiments of this application. The adversarial sample generation device and the image search device that execute the above face image processing method will be described below.

[0176] See Figure 7 ,like Figure 7 The schematic diagram of an adversarial example generation device 40 shown illustrates its ability to perform adversarial perturbation processing on images to be published and previously published, thereby generating adversarial facial examples. This prevents search engines based on neural network models from directly identifying the target facial image matching the specific user even after crawling facial images of a specific user from the web (e.g., crawling previously published facial images of the target user from a business server) and preprocessing the facial images. The adversarial example generation device 40 in this embodiment can achieve the above-mentioned... Figures 1-6b The steps of the face image processing method performed by the adversarial example generation device in any corresponding embodiment. The functions implemented by device 40 can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The device 40 may include an input / output module 401 and a processing module 402. The functional implementation of the input / output module 401 and the processing module 402 can be found in [reference needed]. Figure 1 The operations performed in any of the corresponding embodiments in Figure 6 will not be described in detail here.

[0177] In some implementations, the input / output module 401 can be used to acquire a target face image to which the adversarial face example to be generated is located;

[0178] The processing module 402 can be used to divide the target face image obtained by the input / output module 401 into regions to obtain at least one first region and at least one second region, wherein the first region is a non-facial key feature region and the second region is a facial key feature region.

[0179] At least one first region is subjected to a first perturbation, and at least one second region is subjected to a second perturbation to generate adversarial face examples; the first perturbation and the second perturbation are perturbations with different norms.

[0180] The input / output module 401 is also used to publish or update specific event information, which includes at least one adversarial face sample obtained by the processing module 402.

[0181] In some embodiments, there are multiple target face images. After the input / output module 401 obtains the target face images to be used for generating adversarial examples, and before generating the adversarial examples, the processing module 402 is further configured to:

[0182] Multiple face images are divided into multiple first clusters;

[0183] Multiple first-type annotation regions and multiple second-type annotation regions are displayed on the face images in each first cluster. The first-type annotation regions are used to locate the regions on each face image to which the first perturbation is to be added, and the second-type annotation regions are used to locate the regions on each face image to which the second perturbation is to be added.

[0184] A first perturbation is added in batches to the first type of labeled regions of the face images in each first cluster, and a second perturbation is added in batches to the second type of labeled regions of the face images in each cluster, so as to obtain multiple face adversarial samples.

[0185] In some embodiments, the processing module 402 is specifically used for:

[0186] Determine the feature information of the target features in each first region;

[0187] Obtain the first perturbation that matches the feature information of the target features in each first region;

[0188] The first perturbation is added to the region to which the corresponding target feature belongs.

[0189] In some embodiments, the processing module 402 is specifically used for:

[0190] A first perturbation is applied to at least one of the first regions;

[0191] The multiple face images with the first perturbation added are classified to obtain multiple second clusters;

[0192] The second perturbation is applied to at least one of the second regions of each face image in each second cluster to obtain the face adversarial sample.

[0193] In some embodiments, the processing module 402 is further configured to:

[0194] The input / output module 401 receives a first message from the user terminal. The first message includes specific event information to be published by the target user, and the specific event information includes at least one face image.

[0195] The face images that meet the preset privacy protection conditions in the specific event information are set as the target face images.

[0196] In some embodiments, the processing module 402 is specifically used for:

[0197] Identify target users who meet privacy protection requirements;

[0198] The input / output module 401 obtains the historical image set published by the target user within a historical time period, and the historical image set includes at least one unprocessed candidate historical image of the target user.

[0199] The candidate historical images that meet the preset privacy protection conditions are set as the target face images.

[0200] In some embodiments, the processing module 402 is further configured to:

[0201] The input / output module 401 receives a second message from the user terminal, the second message indicating that at least one perturbation should be added to at least one face image in the specific event information to be published by the target user;

[0202] According to the instructions of the second message, the face image that meets the preset privacy protection conditions in the specific event information is set as the target face image.

[0203] In this embodiment, after the input / output module 401 acquires the target face image, the processing module 402 first divides the feature points in the face image into regions according to non-facial key features and facial key features, obtaining at least one first region and at least one second region. Since the first region is a non-facial key feature region and the second region is a facial key feature region, when generating adversarial examples of faces later, a first perturbation and a second perturbation are added according to the features contained in the first region and the second region, respectively. Since the first perturbation and the second perturbation are perturbations with different norms and large differences, when the first perturbation is added to the first region, the appearance of the first region after adding the first perturbation is more in line with the original features in the first region compared to the uniform addition of perturbation in the prior art. Similarly, when the second perturbation is added to the second region, the appearance of the second region after adding the second perturbation is more in line with the original features in the second region compared to the uniform addition of perturbation in the prior art. In short, by adding different perturbations to different regions of the target face image, the final generated adversarial sample can ensure the attack effect without affecting the appearance of the original facial features on the target face image. This can reduce viewer discomfort or make the sample more subtle and less noticeable, thereby improving the user experience.

[0204] See Figure 8 ,like Figure 8The diagram shows the structure of an image search device 40, which can be applied to crawl facial images of a specific user from the web (e.g., crawling facial images historically posted by the target user from a business server), and preprocessing the facial images to identify target facial images matching the specific user. The adversarial example generation device 40 in this embodiment can achieve the above-mentioned... Figures 1-6b The steps of the face image processing method performed by the adversarial example generation device in any corresponding embodiment. The functions implemented by device 40 can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The device 40 may include an input / output module 401 and a processing module 402. The functional implementation of the input / output module 401 and the processing module 402 can be referred to Figures 1-6b The operations performed in any of the corresponding embodiments will not be described in detail here.

[0205] In some embodiments, a search engine that implements the face image acquisition method described in the second aspect above includes:

[0206] The transceiver module 501 is used to receive search instructions, which include key information of the target user;

[0207] Processing module 502 is configured to, in response to the search instruction received by the transceiver module, obtain a set of candidate images matching the key information from the business server through the transceiver module; wherein, the set of candidate images includes at least one adversarial face sample, the adversarial face sample is generated based on adding a first perturbation to at least one first region and adding a second perturbation to at least one second region, the first region and the second region are obtained by dividing the target face image corresponding to the target user into regions; the first region is a non-key facial feature region, and the second region is a key facial feature region; the first perturbation and the second perturbation are perturbations with different norms;

[0208] The processing module is further configured to determine a target image set from the candidate image set as the search results for the target user, wherein the target image set does not include adversarial face samples.

[0209] The adversarial example generation device 40 for performing the face image processing method and the image search device 50 for performing the face image acquisition method in the embodiments of this application have been described above from the perspective of modular functional entities. The following description will focus on hardware processing aspects of the adversarial example generation device 40 for performing the face image processing method and the image search device 50 for performing the face image acquisition method in the embodiments of this application. It should be noted that in the embodiments of this application… Figure 7In the illustrated embodiment, the physical device corresponding to the input / output module 401 can be an input / output unit, transceiver, radio frequency circuit, communication module, and output interface, etc., and the physical device corresponding to the processing module 402 can be a processor. Figure 7 The adversarial sample generation device 40 shown can have, for example: Figure 9 The structure shown, when Figure 7 The adversarial sample generation device 40 shown has, for example: Figure 9 When the structure shown is used, Figure 9 The processor and transceiver in the device can perform the same or similar functions as the input / output module 401 and processing module 402 provided in the aforementioned embodiment of the adversarial sample generation device 40. Figure 9 The memory stores the computer programs that the processor needs to call when executing the above face image processing method.

[0210] It should be noted that, in the embodiments of this application Figure 8 In the embodiments shown, the physical device corresponding to the transceiver module 501 can be an input / output unit, transceiver, radio frequency circuit, communication module, and output interface, etc., and the physical device corresponding to the processing module 502 can be a processor. Figure 8 The image search device 50 shown can have, for example: Figure 9 The structure shown, when Figure 8 The image search device 50 shown has the following features: Figure 9 When the structure shown is used, Figure 9 The processor and transceiver in the device can perform the same or similar functions as the transceiver module 501 and processing module 502 provided in the aforementioned device embodiment corresponding to the image search device 40. Figure 9 The memory in the processor stores the computer programs that need to be called when executing the above face image acquisition method.

[0211] This application also provides another adversarial example generation device, such as... Figure 10 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The adversarial example generation device can be any adversarial example generation device, including mobile phones, tablets, personal digital assistants (PDAs), point-of-sales (POS) devices, in-vehicle computers, etc. Taking a mobile phone as an example:

[0212] Figure 10 The diagram shown is a block diagram of a portion of the structure of a mobile phone related to the adversarial sample generation device provided in the embodiments of this application. (Reference) Figure 10The mobile phone includes: a radio frequency (RF) circuit 710, a memory 720, an input unit 730, a display unit 740, a sensor 780, an audio circuit 760, a wireless-fidelity (Wi-Fi) module 7100, a processor 780, and a power supply 790, among other components. Those skilled in the art will understand that... Figure 7 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0213] The following is combined Figure 10 A detailed introduction to each component of a mobile phone:

[0214] The RF circuit 710 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 780; additionally, it transmits uplink data to the base station. Typically, the RF circuit 710 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 710 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0215] The memory 720 can be used to store software programs and modules. The processor 780 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, picture playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0216] The input unit 730 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732. The touch panel 731, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 731), and drive the corresponding connected devices according to a pre-set program. Optionally, the touch panel 731 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 780, and can also receive and execute commands sent by the processor 780. In addition, the touch panel 731 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 731, the input unit 730 may also include other input devices 732. Specifically, other input devices 732 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0217] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 740 may include a display panel 741, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 741. Further, a touch panel 731 may cover the display panel 741. When the touch panel 731 detects a touch operation on or near it, it transmits the information to the processor 780 to determine the type of touch event. Subsequently, the processor 780 provides corresponding visual output on the display panel 741 based on the type of touch event. Although in Figure 7, the touch panel 731 and the display panel 741 are shown as two separate components to implement the input and output functions of the mobile phone, in some embodiments, the touch panel 731 and the display panel 741 can be integrated to achieve the input and output functions of the mobile phone.

[0218] The mobile phone may also include at least one sensor 780, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 741 according to the ambient light level, and the proximity sensor can turn off the display panel 741 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0219] Audio circuit 760, speaker 761, and microphone 762 provide an audio interface between the user and the mobile phone. Audio circuit 760 converts received audio data into electrical signals and transmits them to speaker 761, where speaker 761 converts them into sound signals for output. On the other hand, microphone 762 converts collected sound signals into electrical signals, which are received by audio circuit 760, converted into audio data, and then processed by processor 780 before being transmitted via RF circuit 710 to, for example, another mobile phone, or the audio data can be output to memory 720 for further processing.

[0220] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 7100, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 9 The Wi-Fi module 7100 is shown, but it is understood that it is not a necessary component of the mobile phone and can be omitted as needed without changing the nature of the application.

[0221] The processor 780 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 720, and calls data stored in the memory 720 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 780.

[0222] The mobile phone also includes a power supply 790 (such as a battery) that supplies power to various components. The power supply can be logically connected to the processor 780 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0223] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0224] In this embodiment of the application, the processor 780 included in the mobile phone also has the function of controlling and executing the above-mentioned... Figure 10 The method flow shown is executed by the adversarial example generation device 40. The steps performed by the adversarial example generation device in the above embodiments can be based on this... Figure 10 The mobile phone structure is shown. For example, the processor 722 performs the following operations by calling instructions from memory 732:

[0225] The target face image to be generated for the adversarial face example is obtained through the input unit 730.

[0226] The acquired target face image is divided into regions to obtain at least one first region and at least one second region, wherein the first region is a non-facial key feature region and the second region is a facial key feature region.

[0227] At least one first region is subjected to a first perturbation, and at least one second region is subjected to a second perturbation to generate adversarial face examples; the first perturbation and the second perturbation are perturbations with different norms.

[0228] Specific event information is published or updated via RF circuit 710, the specific event information including at least one adversarial face sample obtained by processing module 402.

[0229] In other embodiments of this application, the processor 780 included in the mobile phone also has the function of controlling the execution of the above-mentioned components. Figure 8 The method flow executed by the image search device 50 shown is illustrated. The steps performed by the adversarial example generation device in the above embodiments can be based on this... Figure 10 The mobile phone structure is shown. For example, the processor 722 performs the following operations by calling instructions from memory 732:

[0230] The input unit 730 receives a search instruction, which includes key information about the target user.

[0231] In response to the search command, a set of candidate images matching the key information is obtained from the business server via the input unit 730; wherein, the set of candidate images includes at least one adversarial face sample, which is generated based on adding a first perturbation to at least one first region and adding a second perturbation to at least one second region, wherein the first region and the second region are obtained by dividing the target face image corresponding to the target user into regions; the first region is a non-key facial feature region, and the second region is a key facial feature region;

[0232] The target image set is determined from the candidate image set as the search results for the target user, and the target image set does not include adversarial face samples.

[0233] This application also provides another face adversarial example generation device for implementing the above-described face image processing method, or a search device for implementing the above-described face image acquisition method, such as... Figure 11 As shown, Figure 11 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1020 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1022 (e.g., one or more processors) and memory 1032, and one or more storage media 1030 (e.g., one or more mass storage devices) for storing application programs 1042 or data 1044. The memory 1032 and storage media 1030 can be temporary or persistent storage. The program stored in the storage media 1030 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 1022 may be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1030 on the server 1020.

[0234] Server 1020 may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, and / or one or more operating systems 1041, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0235] In the above embodiments, the service server (e.g.) Figure 7 The face adversarial example generation device 40 shown) or search engine (e.g. Figure 8 The steps performed by the search device 50 shown can be based on this Figure 11 The structure of server 1020 is shown. For example, in the above embodiment, it consists of... Figure 7 The steps performed by the face adversarial example generation device 40 shown can be based on this Figure 11 The server architecture is shown. For example, the processor 1022 performs the following operations by calling instructions from memory 1032:

[0236] The target face image to be generated for the adversarial face example is obtained through the input / output interface 1058.

[0237] The target face image is divided into regions to obtain at least one first region and at least one second region, wherein the first region is a non-facial key feature region and the second region is a facial key feature region.

[0238] At least one first region is subjected to a first perturbation, and at least one second region is subjected to a second perturbation to generate adversarial face examples; the first perturbation and the second perturbation are perturbations with different norms.

[0239] Publish or update specific event information through input / output interface 1058, the specific event information including at least one face adversarial sample.

[0240] For example, in the above embodiments, by Figure 8 The steps performed by the search device 50 shown can be based on this Figure 11 The server architecture is shown. For example, the processor 1022 performs the following operations by calling instructions from memory 1032:

[0241] The search command is received through the input / output interface 1058, and the search command includes key information of the target user.

[0242] In response to the search command, a set of candidate images matching the key information is obtained from the business server; wherein, the set of candidate images includes at least one adversarial face sample, which is generated based on adding a first perturbation to at least one first region and adding a second perturbation to at least one second region, wherein the first region and the second region are obtained by dividing the target face image corresponding to the target user into regions; the first region is a non-key facial feature region, and the second region is a key facial feature region;

[0243] The target image set is determined from the candidate image set as the search results for the target user, and the target image set does not include adversarial face samples.

[0244] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0245] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0246] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0247] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0248] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0249] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0250] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0251] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A method for processing human face images, characterized in that, The method is applied to a business server, and the method includes: Obtain the target face image from which the adversarial face example to be generated; The target face image is divided into regions to obtain at least one first region and at least one second region, wherein the first region is a non-facial key feature region and the second region is a facial key feature region. At least one first region is subjected to a first perturbation, and at least one second region is subjected to a second perturbation to generate adversarial face examples; the first perturbation and the second perturbation are perturbations with different norms. Publish or update specific event information, wherein the specific event information includes at least one of the face adversarial samples; Wherein, there are multiple target face images, and after obtaining the target face images to be used to generate adversarial examples, but before generating the adversarial examples, the method further includes: Multiple face images are divided into multiple first clusters; The step of adding a first perturbation to at least one first region and a second perturbation to at least one second region to generate adversarial face examples includes: Multiple first-type annotation regions and multiple second-type annotation regions are displayed on the face images in each first cluster. The first-type annotation regions are used to locate the regions on each face image to which the first perturbation is to be added, and the second-type annotation regions are used to locate the regions on each face image to which the second perturbation is to be added. A first perturbation is added in batches to the first type of labeled regions of the face images in each first cluster, and a second perturbation is added in batches to the second type of labeled regions of the face images in each cluster, so as to obtain multiple face adversarial samples.

2. The method according to claim 1, characterized in that, The first perturbation applied to at least one first region includes: Determine the feature information of the target features in each first region; Obtain the first perturbation that matches the feature information of the target features in each first region; The first perturbation is added to the region to which the corresponding target feature belongs.

3. The method according to claim 1, characterized in that, The step of adding a first perturbation to at least one first region and a second perturbation to at least one second region to generate adversarial face examples includes: A first perturbation is applied to at least one of the first regions; The multiple face images with the first perturbation added are classified to obtain multiple second clusters; The second perturbation is applied to at least one of the second regions of each face image in each second cluster to obtain the face adversarial sample.

4. The method according to any one of claims 1-3, characterized in that, The process of obtaining the target face image to be used to generate adversarial examples includes: Receive a first message from the user terminal, the first message including specific event information to be published by the target user, the specific event information including at least one face image; The face images that meet the preset privacy protection conditions in the specific event information are set as the target face images.

5. The method according to any one of claims 1-3, characterized in that, The process of obtaining the target face image to be used to generate adversarial examples includes: Identify target users who meet the preset privacy protection conditions; Obtain the historical image set published by the target user within a historical time period, the historical image set including at least one unprocessed candidate historical image of the target user; The candidate historical images that meet the preset privacy protection conditions are set as the target face images.

6. The method according to any one of claims 1-3, characterized in that, The process of obtaining the target face image to be used to generate adversarial examples includes: Receive a second message from the user terminal, the second message instructing the addition of at least one perturbation to at least one face image in a specific event information to be published by the target user; According to the instructions of the second message, the face image that meets the preset privacy protection conditions in the specific event information is set as the target face image.

7. A processing device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the method as described in any one of claims 1 to 6 when it invokes the computer program in the memory.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the method of any one of claims 1 to 6.

Images