A face image processing method, related device and storage medium

Abstract

Embodiments of the present application relate to the field of data security, and provide a face image processing method, related device and storage medium, the method comprising: a business server acquiring a target face image to be processed; inputting the target face image into a target model to obtain a target adversarial sample, wherein the target model is obtained based on unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold; publishing the target adversarial sample, or updating the target face image to the target adversarial sample. The present scheme can improve the generation efficiency of adversarial samples and protect user privacy data.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data security, and in particular to a face image processing method, related apparatus, and storage medium. BACKGROUND

[0002] With the popularity of the Internet, while people are facilitated to obtain more information, personal privacy information is also more likely to be leaked. For example, in the information system of a modern enterprise, in the case of needing to publicly disclose a personal face image, the publicly disclosed face image is vulnerable to attacks and causes leakage of personal privacy information. In order to resist attacks, encryption is generally performed on the face image to form a face image with a watermark, for example.

[0003] In the research and practice of the prior art, the inventors of the present application found that the encryption method used in the prior art needs to determine some features of a target face image first, and then encrypts the face image in a targeted manner, which results in a slow generation speed of the encrypted image. SUMMARY

[0004] Embodiments of the present application provide a face image processing method, related apparatus, and storage medium, which can improve the generation efficiency of adversarial samples.

[0005] In a first aspect, embodiments of the present application provide a face image processing method from the perspective of a business server, which includes:

[0006] obtaining a target face image to be processed;

[0007] inputting the target face image into a target model to obtain a target adversarial sample, wherein the target model is obtained based on unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold;

[0008] publishing the target adversarial sample, or updating the target face image to the target adversarial sample.

[0009] In some embodiments, the plurality of sub-images includes at least one first region and at least one second region, wherein the first region is a non-face key feature region, and the second region is a face key feature region.

[0010] In some embodiments, the target face image is an initial face image obtained.

[0011] In some embodiments, the normalization mapping processing of the target feature vector to obtain the target adversarial sample includes:

[0012] The initial adversarial sample is subjected to augmentation processing to obtain the target adversarial sample, and the augmentation processing includes rotation and compression processing on the initial adversarial sample.

[0013] In some embodiments, the target face image is obtained from at least one of the following manners:

[0014] The target user has specific event information to be published on a social platform, and the specific event information includes at least one initial face image;

[0015] Alternatively, the target user has a historical image set published on a social platform in a historical period, and the historical image set includes at least one unprocessed candidate historical image of the target user, and the candidate historical image includes an image of the target user meeting a preset privacy protection condition.

[0016] In some embodiments, the target face image is obtained from at least one of the following manners:

[0017] The target user has a historical image set to be uploaded on a financial platform or a government platform;

[0018] Alternatively, the target user has a historical image set uploaded on a financial platform or a government platform in a historical period;

[0019] The historical image set includes at least one unprocessed candidate historical image of the target user.

[0020] In some embodiments, the target face image to be processed is obtained by:

[0021] The business server determines the number of inputs of the target model, and inputs each target face image as an input according to the number of inputs to obtain a target adversarial sample corresponding to each target face image, wherein each input corresponds to a pixel region in the target face image.

[0022] In a second aspect, an embodiment of the present application provides a face image processing method from the perspective of a search engine, and the method comprises:

[0023] Receiving a search instruction, the search instruction including key information of a target user;

[0024] In response to the search instruction, a candidate image set matching the key information is obtained from a business server; wherein the candidate image set includes at least one target adversarial sample, the target adversarial sample being obtained by inputting a target face image into a target model; the target model being obtained by unsupervised learning based on unsupervised learning; and the similarity between the target adversarial sample and the target face image being lower than a preset threshold;

[0025] determine a target image set as a search result of the target user from the candidate image set, the target image set not including a target adversarial sample.

[0026] In some embodiments, the target image set refers to at least one image published by at least one target user and including at least one face.

[0027] In a third aspect, the embodiments of the present application provide an image processing apparatus having a function of implementing the face image processing method provided in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware.

[0028] In some embodiments, the image processing apparatus includes:

[0029] an input and output module configured to obtain a target face image to be processed;

[0030] a processing module configured to input the target face image to a target model to obtain a target adversarial sample, wherein the target model is obtained based on unsupervised learning, and a similarity between the target adversarial sample and the target face image is lower than a preset threshold;

[0031] The input and output module is further configured to publish the target adversarial sample obtained by the processing module, or update the target face image to the target adversarial sample by the processing module.

[0032] In some embodiments, the processing module is specifically configured to:

[0033] perform nonlinear mapping processing on the target face image in a convolution layer to obtain a high-dimensional feature vector;

[0034] perform nonlinear mapping processing on the high-dimensional feature vector in a deconvolution layer to obtain a target feature vector;

[0035] perform normalization mapping processing on the target feature vector to obtain the target adversarial sample.

[0036] In some embodiments, the processing module is specifically configured to:

[0037] perform augmentation processing on the initial adversarial sample to obtain the target adversarial sample, and the augmentation processing includes rotation and compression processing on the initial adversarial sample.

[0038] In some embodiments, before inputting the target face image to the target model, the processing module is further configured to:

[0039] obtaining a training set and the face recognition model to be trained through the input and output module, the training set comprising a plurality of face images;

[0040] inputting the training set into the face recognition model to obtain an adversarial sample set comprising a plurality of adversarial samples;

[0041] comparing each adversarial sample in the adversarial sample set with a corresponding face image in the training set, and if a loss value obtained by a loss function of the face recognition model is less than a preset threshold, inputting the adversarial sample set into the face recognition model to train and update the face recognition model to obtain the target model.

[0042] In some embodiments, the loss function is obtained according to the following expression:

[0043]

[0044] wherein x is a face image in the training set, x1 is an adversarial sample in the adversarial sample set, denotes a feature vector output by the face recognition model;

[0045] The preset threshold satisfies the following inequality:

[0046] ||x-x′|| ∞ ≤∈

[0047] wherein ∈ is a selected perturbation size.

[0048] In some embodiments, the processing module is specifically configured to:

[0049] obtaining an initial face image through the input and output module;

[0050] dividing the initial face image into regions to obtain a plurality of sub-images;

[0051] using part or all of the plurality of sub-images as the target face image.

[0052] In some embodiments, the plurality of sub-images comprises at least one first region and at least one second region, wherein the first region is a non-face key feature region, and the second region is a face key feature region.

[0053] In some embodiments, the target face image is the obtained initial face image.

[0054] In some embodiments, the processing module is specifically configured to:

[0055] receiving, by the input / output module, a first message from a user terminal, the first message comprising specific event information to be published by a target user to a social platform, or to be uploaded by the target user to a financial platform or a government platform, the specific event information comprising at least one initial face image;

[0056] setting, as the target face image, an initial face image in the specific event information that meets a preset privacy protection condition.

[0057] In some embodiments, the processing module is specifically configured to:

[0058] determining a target user that meets a preset privacy protection condition;

[0059] obtaining a historical image set published by the target user to a social platform, or uploaded by the target user to a financial platform or a government platform within a historical period, the historical image set comprising at least one unprocessed candidate historical image of the target user;

[0060] setting, as the target face image, a candidate historical image in the candidate historical image that meets the preset privacy protection condition.

[0061] In some embodiments, the target face image is derived from at least one of the following manners:

[0062] specific event information to be published by a target user to a social platform, the specific event information comprising at least one initial face image;

[0063] or, a historical image set published by a target user to a social platform within a historical period, the historical image set comprising at least one unprocessed candidate historical image of the target user, the candidate historical image comprising an image of the target user that meets a preset privacy protection condition.

[0064] In some embodiments, the target face image is derived from at least one of the following manners:

[0065] a historical image set to be uploaded by a target user to a financial platform or a government platform;

[0066] or, a historical image set uploaded by a target user to a financial platform or a government platform within a historical period;

[0067] wherein the historical image set comprises at least one unprocessed candidate historical image of the target user.

[0068] In some embodiments, the processing module is specifically configured to:

[0069] receive a second message from a user terminal through the input and output module, the second message indicating that at least one face image in specific event information to be published by a target user to a social platform or to be uploaded to a financial platform or a government platform is to be subjected to privacy protection processing;

[0070] determine, according to the indication of the second message, a face image in the specific event information that meets a preset privacy protection condition as the target face image.

[0071] In some embodiments, the processing module is specifically configured to:

[0072] determine the number of inputs of the target model;

[0073] input sub-images in the target face image into the target model as one input respectively according to the number of inputs, to obtain the target adversarial sample, wherein each input corresponds to a pixel region in the target face image.

[0074] In a fourth aspect, the embodiments of the present application further provide a search engine for implementing the face image acquisition method in the second aspect, and the search engine comprises:

[0075] a transceiving module configured to receive a search instruction, the search instruction comprising key information of a target user;

[0076] a processing module configured to, in response to the search instruction, acquire, through the transceiving module, a candidate image set matching the key information from a service server; wherein the candidate image set comprises at least one target adversarial sample, the target adversarial sample being obtained by inputting a target face image into a target model; the target model being obtained based on unsupervised learning; and the similarity between the target adversarial sample and the target face image being lower than a preset threshold.

[0077] The processing module is further configured to determine, from the candidate image set, a target image set as a search result of the target user, the target image set not comprising the target adversarial sample.

[0078] In some embodiments, the target face image is at least one image published by at least one target user and comprising at least one face.

[0079] In a fifth aspect, the embodiments of the present application provide a face image processing apparatus, comprising at least one processor and a memory; wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program stored in the memory to execute the steps in the face image processing method in the first aspect, any one of the embodiments of the first aspect, or any one of the face image processing methods provided in the second aspect.

[0080] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having a function of implementing the face image processing method corresponding to the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, which can be software and / or hardware. Specifically, the computer-readable storage medium stores a plurality of instructions, which are suitable for being loaded by a processor to execute the steps in the first aspect, any of the implementation manners of the first aspect, or any of the face image processing methods provided by the second aspect.

[0081] Compared with the prior art, in the scheme provided by the embodiment of the present application, since the target model is obtained based on unsupervised learning, the similarity between the image output by the target model (i.e., the target adversarial sample) and the target face image is lower than the preset threshold, that is, the target adversarial sample obtained by processing the target face image by the target model is different from the target face image before being processed by the target model. Therefore, in one aspect, when the target adversarial sample is published on the business server (such as a social platform), even if the search engine crawls the target adversarial sample, the target adversarial sample will be misidentified due to the adversarial nature of the target adversarial sample, or the target adversarial sample cannot be bound with the identity information of the target user to be searched, so that the privacy data of the target user can be effectively protected from being maliciously collected or utilized. In another aspect, after the target face image uploaded on the business server (such as a social platform, a financial platform, a government platform, or other platforms requiring to upload privacy images including the face of the target user) is updated to the target adversarial sample, even if the target adversarial sample uploaded to the business server is leaked due to some reasons, or is crawled by the search engine after being leaked, the target adversarial sample will be misidentified due to the adversarial nature of the target adversarial sample, or the target adversarial sample cannot be bound with the identity information of the target user to be searched, so that the privacy data of the target user can be effectively protected from being maliciously collected or utilized.

[0082] Furthermore, since the target model is obtained through unsupervised learning, compared to existing technologies that require "extracting facial features and undergoing multiple iterations to obtain adversarial samples that are the same as or similar to the target adversarial sample," this embodiment does not require facial feature extraction from the target face image. It only requires a pre-trained target model based on unsupervised learning, and then inputting the real-time acquired target face image into the target model. The target model can process the image in one go without iteration, quickly obtaining a target adversarial sample corresponding to the target face image with a similarity below a preset threshold. This target adversarial sample does not cause visual interference for the user; it appears as the same image to the user. However, for machine vision, this target adversarial sample cannot be directly recognized by an intelligent search engine based on an image recognition model and cannot be bound to the target user's identity information. Therefore, this embodiment can effectively protect the target user's privacy data while significantly improving the efficiency of generating the target adversarial sample, thereby improving the processing efficiency of the business server and reducing its workload. Attached Figure Description

[0083] Figure 1 This is a schematic diagram of an application environment provided in the embodiments of this application;

[0084] Figure 2 This is a schematic flowchart of a model training method in an embodiment of this application.

[0085] Figure 3 This is a flowchart illustrating a face image processing method in an embodiment of this application.

[0086] Figure 4a This is a schematic diagram of a model structure for generating target adversarial examples in an embodiment of this application;

[0087] Figure 4b This is a schematic diagram of adding adversarial perturbations to a target face image to obtain a target adversarial sample in an embodiment of this application;

[0088] Figure 4c This is a schematic diagram illustrating the addition of perturbation to the facial image of the target user to be posted on Weibo in an embodiment of this application;

[0089] Figure 4d This is a schematic diagram illustrating the addition of perturbation to a complete face image synthesized from a nine-grid image uploaded to a social media platform, as described in this application embodiment.

[0090] Figure 4e This is a schematic diagram illustrating the addition of perturbation to the facial images of target users historically posted on social media platforms, as described in this application embodiment.

[0091] Figure 4f is a schematic diagram of adding a disturbance to a face picture of a target user and a specific user historically published on a social platform in an embodiment of the present application;

[0092] Figure 5 is a flowchart of an image acquisition method in an embodiment of the present application;

[0093] Figure 6 is a scene schematic diagram of an image acquisition method in an embodiment of the present application;

[0094] Figure 7 is a structural schematic diagram of an image processing device in an embodiment of the present application;

[0095] Figure 8 is a structural schematic diagram of a search device in an embodiment of the present application;

[0096] Figure 9 is a structural schematic diagram of an entity device implementing a face image processing method in an embodiment of the present application;

[0097] Figure 10 is a structural schematic diagram of a mobile phone implementing a face image processing method in an embodiment of the present application;

[0098] Figure 11 is a structural schematic diagram of a server implementing a face image processing method in an embodiment of the present application. DETAILED DESCRIPTION

[0099] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects (e.g., the first region and the second region in the embodiments of this application represent different regions in the initial face image), and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The module divisions appearing in the embodiments of this application are merely logical divisions; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. In addition, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces, and the indirect couplings or communication connections between modules may be electrical or other similar forms, none of which are limited in the embodiments of this application. Moreover, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0100] This application provides a face image processing method, related apparatus, and storage medium, which can be used on servers or terminal devices. Specifically, it can be used to perform privacy protection processing on images to be published or previously published, thereby generating face adversarial examples. This makes it impossible for a search engine (i.e., a search device) based on a neural network model to directly identify the target face image matching the specific user even after crawling face images of a specific user from the network (e.g., crawling face images previously published by the target user from a business server) and preprocessing the face images.

[0101] In some implementations, this solution is applied to, for example... Figure 1 The communication system shown may include at least one service server and at least one search device. Figure 1 Only one search device (a) is shown in the diagram. This search device can be a server or a terminal. When it is a terminal, it has a client installed. The clients installed on each search device can be the same or different versions of the client, or clients with the same or different operating systems. This application does not limit the version of the client or the type of operating system corresponding to the client. A detailed description follows.

[0102] Business server: can be used to provide image search to each search device, and provide image upload, download or preprocessing and other business services to users, for example, the server can perform adversarial perturbation processing on the image uploaded from the user, and then update online. Image processing devices can be deployed in the business server.

[0103] Search device: can crawl face images or videos matching the key information of the target user from the business server, then pre-process the images or videos, and identify the pre-processed images to search out images including the face of the target user.

[0104] When the above face image processing method and face image acquisition method are implemented based on the communication system as shown in Figure 1 , the business server can perform privacy protection processing on the face image a uploaded by the target user (for example, input the face image a into a target model based on unsupervised learning to add adversarial perturbation to the face image a to obtain a target adversarial sample), and then publish the target adversarial sample after online updating. On the one hand, this makes the search device unable to identify that the face image a belongs to the target user when the search device crawls the face image a from the business server, that is, the face recognition function of the search device is interfered. On the other hand, since the business server adds small perturbations to the cheeks and forehead and large perturbations to other positions in the process of generating the face adversarial sample, the influence of the adversarial perturbation on the visual effect of the face can be effectively reduced.

[0105] The scheme provided by the embodiments of the present application relates to technologies such as artificial intelligence (AI), nature language processing (NLP), machine learning (ML), and the like, and is specifically explained as follows:

[0106] AI is a theory, method, technology and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0107] AI technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes, such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other directions.

[0108] NLP is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, that is, the language used in daily life, so it has a close relationship with the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies.

[0109] For the face recognition direction in the field of artificial intelligence, the application can generate a general perturbation with high attack robustness by using a face image set, and by adding a general perturbation to the face image in the face image set, the image classification model cannot accurately recognize the face image after adding the perturbation, thereby realizing the attack on the image classification model.

[0110] Among them, it needs to be particularly pointed out that the server (such as a business server, a search engine) involved in the embodiments of the application can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The image processing device involved in the embodiments of the application can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a personal digital assistant, etc., but is not limited thereto. The image processing device and the server can be directly or indirectly connected through wired or wireless communication, and the embodiments of the application do not limit this.

[0111] When a user's portrait is spread on a public social network, the user does not want a third party to search for the user's own face image by means of a crawler, a search engine, etc., using AI image recognition technology represented by face recognition, so he needs to add adversarial perturbation to the image to interfere with face recognition. At the same time, the user wants his portrait to be watched in the friend circle, and does not affect the viewer's judgment of the user's appearance, so the user does not want the adversarial perturbation to excessively affect the visual effect of his appearance. To this end, the embodiments of the application mainly adopt the following technical solutions:

[0112] 1. Input the images to be published and the images previously published into the target model pre-trained based on unsupervised learning to perform privacy protection processing on the images to be published and the images previously published, thereby generating corresponding target adversarial examples. This makes it impossible for a search engine based on a neural network model to directly identify the target face image matching the specific user even after crawling the face image of a specific user from the network (e.g., crawling the face image previously published by the target user from the business server) and preprocessing the face image.

[0113] 2. Directly input the training set into the face recognition model, and pre-train the face recognition model based on the unsupervised model. There is no need to extract the facial features of the training images in the training set, so as to obtain the target model that can quickly generate target adversarial examples.

[0114] The following combination Figures 2-10 The technical solutions of the embodiments of this application will be described by way of example.

[0115] Since the face image processing method in this application is based on a pre-trained target model to process face images, the training process of the target model will be introduced before introducing the face image processing method. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of a training process for a target model, which includes:

[0116] 101. Obtain the training set and the face recognition model to be trained.

[0117] The training set includes multiple face images.

[0118] 102. Input the training set into the face recognition model to obtain the adversarial example set.

[0119] The adversarial sample set includes multiple adversarial samples.

[0120] 103. Compare each adversarial sample in the adversarial sample set with the corresponding face image in the training set. If the loss value obtained by the loss function of the face recognition model is less than a preset threshold, then input the adversarial sample set into the face recognition model to train and update the face recognition model to obtain the target model.

[0121] In some implementations, the loss function can be obtained from the following expression:

[0122]

[0123] Where x represents a face image in the training set, and x1 represents an adversarial example in the adversarial example set. a feature vector output by the face recognition model;

[0124] The preset threshold satisfies the following inequality:

[0125] ||x-x′| ∞ ≤∈

[0126] wherein, ∈ is a selected perturbation size.

[0127] In some embodiments, the target model can be a generative adversarial network (GAN), Figure 4a A structure diagram of the target model, and the selection and structure of the model are not limited in the embodiments of the present application.

[0128] In the embodiments of the present application, since the target model is obtained based on unsupervised learning, specifically, in the training phase, the face feature of the face image in the training set does not need to be extracted, and the training set is directly input into the face recognition model to obtain the adversarial sample set. On the one hand, the efficiency of the adversarial sample set obtained in this way is very high, thereby improving the efficiency of training the face recognition model based on the adversarial sample set; on the other hand, since the adversarial samples in the adversarial sample set are compared with the corresponding face images in the training set before the face recognition model is trained, when the loss value obtained by the loss function of the face recognition model is less than the preset threshold, the adversarial sample set is input into the face recognition model to train and update the face recognition model based on the adversarial sample set, and the target model is obtained. Therefore, after the target face image is obtained, it can be processed in the target model without iteration, and the target adversarial sample corresponding to the target face image and having a similarity lower than the preset threshold can be quickly obtained, and the target adversarial sample does not cause visual recognition interference to the user, and for the user, it is the same image, but for machine vision, the target adversarial sample cannot be directly recognized by the intelligent search engine based on the image recognition model, and is bound to the identity information of the target user. As can be seen, the embodiments of the present application can well support the subsequent protection of the privacy data of the target user, and can significantly improve the efficiency of generating the target adversarial sample, thereby improving the processing efficiency of the business server and reducing the work load of the business server.

[0129] As described in the above embodiments, after the target model is obtained by model training, the face image to be processed can be processed based on the target model to obtain an adversarial sample meeting the privacy protection condition. Specifically, referring to Figure 3Provided is a face image processing method based on an unsupervised learning target model. Taking adding a watermark in a face image for reverse search as an example, the method can be executed by a business server side, which can be a social platform, a financial platform, a government affair platform, a search system or the like that needs to upload a private image including a target user face, and embodiments of the present application are not limited in this regard. Embodiments of the present application include:

[0130] 201, obtaining a target face image to be processed.

[0131] The target face image is a face image that needs to be protected, including but not limited to a face image of a target user himself / herself, a face image of a specific user. The specific user includes a face image of a person having a correlation (for example, a friend relationship) with the target user, and can also include a face image of an arbitrary user designated by the target user, a face image of a user satisfying a preset privacy protection condition designated by the business server, and embodiments of the present application are not limited in this regard. The number of target face images input into the target model and the users corresponding to the target face images are not limited in embodiments of the present application.

[0132] In some embodiments, since the business server can provide various business services for users, and the business server can be a social platform, a financial platform, a government affair platform or a search platform, the target face images that can be processed by the business server also come from different sources, and specifically, the data leakage or crawling paths can be used for illustration:

[0133] (1) For crawling a target face image published by a user on a social platform

[0134] In this case, the target face image can come from at least one of the following ways:

[0135] a1, specific event information to be published by the target user on the social platform.

[0136] The specific event information includes at least one initial face image.

[0137] For example, user a is editing a microblog a on Sina Weibo but has not published it, and the microblog a includes multiple face images uploaded by user a. The multiple face images uploaded by user a can include only a face image of user a, only a face image of user b, or both a face image of user a and a face image of user b, and embodiments of the present application are not limited in this regard. A schematic diagram can be referred to as Figure 4c , which is not repeated here.

[0138] b1, a historical image set published by the target user on the social platform in a historical period.

[0139] The historical image set includes at least one unprocessed candidate historical image of the target user, and the candidate historical image includes an image of the target user meeting a preset privacy protection condition.

[0140] For example, user a has published 20 microblogs (referred to as historical microblogs) on Sina Weibo before February 27, 2022. Among the historical microblogs, 2 microblogs (for example, microblog b and microblog c) include multiple face images (referred to as historical face images) uploaded by user a. Similarly, the historical face images uploaded by user a can include only the face image of user a, only the face image of user b, or both the face images of user a and user b. The embodiments of the present application do not limit this. A schematic diagram can be referred to as Figure 4e , and details are not repeated.

[0141] It can be seen that the embodiments of the present application can perform privacy protection processing on images to be published and published in various channels, so that after publishing a face image related to one's own privacy on a social platform, even if the face image is crawled to the local search engine of the search engine based on the face recognition model, it is highly likely to be filtered by the search engine, that is, it can resist the search of the search engine based on the face recognition model to a certain extent.

[0142] (2) For data leaked on a financial platform or a government platform, and target face images crawled from the leaked data

[0143] a2, a target image set to be uploaded by a target user on a financial platform or a government platform.

[0144] The target image set can be a separately uploaded image set or an image set included in identity verification and related materials. The embodiments of the present application do not limit this, and other embodiments are the same, and details are not repeated.

[0145] For example, when user b performs identity binding or identity verification and other businesses on financial platform a, the user b needs to upload at least one face image including multiple face images of the user b on the financial platform a. The multiple face images of the user b can be identity card photos, residence cards, passports, and registration photos.

[0146] b2, a historical image set uploaded by a target user on a financial platform or a government platform in a historical period.

[0147] The historical image set includes at least one unprocessed candidate historical image of the target user.

[0148] It can be seen that the embodiments of the present application can perform privacy protection processing on the images to be published and the published images of various channels, so that after a face image related to the privacy of a user is uploaded on a financial platform or a government platform, even if it is leaked due to some reasons and flows to the network, it is highly likely to be filtered by a search engine, that is, it can resist the search of the search engine based on the face recognition model to a certain extent.

[0149] In some other embodiments, the target face image can be a sub-image in the whole initial face image or an obtained initial face image, that is, the whole initial face image can be input into the target model to generate the target adversarial sample, or a sub-image of the target user in the initial face image can be input into the target model to generate the target adversarial sample. For example, when the initial face image only includes at least one face image of the target user, a sub-image of the face image of the target user is obtained therefrom; or when the initial face image includes the face image of the target user and the face image of another user, the initial face image can be divided to obtain the face image of the target user therefrom. The manner of obtaining the target face image to be processed is as follows:

[0150] obtaining an initial face image;

[0151] dividing the initial face image into regions to obtain a plurality of sub-images;

[0152] using part or all of the plurality of sub-images as the target face image.

[0153] The plurality of sub-images include at least one first region and at least one second region, wherein the first region is a non-face key feature region, such as a forehead, a chin, a cheek, etc. The second region is a face key feature region, such as a brow, an eye, a nose, a mouth, an ear, etc.

[0154] It can be seen that the embodiments of the present application fully consider the user corresponding to the sub-image included in the initial face image, and can effectively improve the subsequent generation of the target adversarial sample based on the preprocessed target face image.

[0155] The embodiments of the present application do not limit the shape, division granularity, division manner, number, area, etc. of the first region and the second region. There are multiple non-face key feature regions and face key feature regions in the face region, and in the embodiments of the present application, the regions can be divided according to actual needs (such as business needs, precision, etc.). In some embodiments, only at least one first region can be divided, that is, a region dedicated to adding a first disturbance, which is not limited.

[0156] 202. Input the target face image into the target model to obtain the target adversarial sample.

[0157] The target model is obtained through unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold.

[0158] A target adversarial example refers to a face image obtained by processing a target face image through a target model. This adversarial example ensures that it does not affect the user's visual perception of the target face image, nor does it affect the visual quality of the target face image. Furthermore, it prevents AI-based search engines from identifying the target user through crawling, thus rendering the crawled adversarial example useless. In some implementations, the target adversarial example can be obtained in the following way (see Figure 4 for the specific process):

[0159] The target face image is subjected to nonlinear mapping processing in the convolutional layer to obtain a high-dimensional feature vector;

[0160] The high-dimensional feature vector is subjected to nonlinear mapping processing in the deconvolution layer to obtain the target feature vector;

[0161] The target feature vector is normalized and mapped to obtain the target adversarial sample.

[0162] 203. Publish the target adversarial sample, or update the target face image with the target adversarial sample.

[0163] The publication of the target adversarial sample can refer to posting moods on a social media platform. For example, a target user posts a Weibo post (a) on Weibo, where the post includes a facial image (a) of the target user and / or a specific user. This facial image (a) is a target adversarial sample obtained after privacy protection processing of the initial facial image (a') uploaded by user a. In other words, the facial image (a') is input into the target model, and the target model outputs the adversarial sample as the aforementioned facial image (a), i.e., replacing facial image (a) with facial image (a'). For example, Figure 4b Based on Figure 4a The diagram shown illustrates how a target model generates adversarial examples.

[0164] Updating the target face image with the target adversarial sample can refer to the following two scenarios (1) and (2):

[0165] (1) After processing the face images that the target user has posted on social media platforms in the past for privacy protection, replace them with the target adversarial sample.

[0166] For example, user a uploads micro-blog b on a micro-blog platform, which includes a face image b of a target user and / or a specific user, and the face image b is a target adversarial sample obtained by privacy protection processing on a user's historically uploaded face image b', i.e., inputting the face image b' into a target model, using the target model output as the above-mentioned face image b, i.e., replacing the face image b with the face image b'. Figure 4d , Figure 4e ,

[0167] In some embodiments, as shown in the micro-blog attachment composed of a nine-square grid diagram in which two images are partial face images of a target user, the business server first locates the partial face images containing the target user (which can also include a specific user) from the nine-square grid micro-blog attachment before inputting these partial face images into a target model to obtain target adversarial samples and then replacing the nine-square grid micro-blog attachment. Figure 4d

[0168] (2) Replacing a target user's historically uploaded face image on a financial platform with a target adversarial sample after privacy protection processing.

[0169] For example, user a uploads an ID image c on a financial platform, which includes an ID image c of a target user and / or a specific user, and the ID image c is a target adversarial sample obtained by privacy protection processing on a user's historically uploaded ID image c', i.e., inputting the ID image c' into a target model, using the target model output as the above-mentioned ID image c, i.e., replacing the ID image c with the ID image c'.

[0170] ​In the embodiment of the present application, since the target model is obtained based on unsupervised learning, the similarity between the image output by the target model (i.e., the target adversarial sample described above) and the target face image is lower than the preset threshold, that is, the target adversarial sample obtained after the target face image is processed by the target model is different from the target face image before being processed by the target model. Therefore, on the one hand, when the target adversarial sample is published on the business server (such as a social platform), even if the search engine crawls the target adversarial sample, the target adversarial sample will be misidentified due to its adversarial nature, or the target adversarial sample cannot be bound to the identity information of the target user to be searched, so as to effectively protect the privacy data of the target user from being maliciously collected or utilized. On the other hand, after the target face image uploaded to the business server (such as a social platform, a financial platform, a government platform, or other platforms that need to upload private images including the face of the target user) is updated to the target adversarial sample, even if the target adversarial sample uploaded to the business server is leaked due to some reasons, or is crawled by the search engine after being leaked, the target adversarial sample will be misidentified due to its adversarial nature, or the target adversarial sample cannot be bound to the identity information of the target user to be searched, so as to effectively protect the privacy data of the target user from being maliciously collected or utilized.

[0171] In addition, since the target model is obtained based on unsupervised learning, compared with the prior art which needs to extract face features and perform multiple iterations and other cumbersome operations to obtain an adversarial sample similar to the target adversarial sample described above, in the embodiment of the present application, the target face image does not need to be extracted for face features. Instead, the target model trained based on unsupervised learning in advance is only needed, and then the target face image acquired in real time is input into the target model. The target model can process the target face image once without iteration, quickly obtain a target adversarial sample corresponding to the target face image and having a similarity lower than the preset threshold. The target adversarial sample does not cause recognition interference in the visual sense of a user, and for the user, it is the same image, but for machine vision, the target adversarial sample cannot be directly recognized by an intelligent search engine based on an image recognition model and bound to the identity information of the target user. Therefore, the embodiment of the present application can effectively protect the privacy data of the target user while significantly improving the efficiency of generating the target adversarial sample (e.g., more than 50 times), thereby improving the processing efficiency of the business server and reducing the work load of the business server.

[0172] Based on the above description of the source of the target face image, the following describes several ways and timing of obtaining the target face image for generating the face adversarial sample, such as initiating a privacy protection requirement from the target user, or the business server initiating a privacy protection operation for the user (e.g., the target user, a specific user), to introduce how to obtain the target face image to be processed:

[0173] (1) Processing scenario of the business server publishing information in real time for a single user based on event triggering

[0174] Specifically, the user terminal sends a first message to the business server, and the business server receives the first message from the user terminal. The first message includes specific event information to be published by the target user to a social platform, or to be uploaded to a financial platform or a government platform, and the specific event information includes at least one initial face image.

[0175] The business server sets the initial face image in the specific event information that meets the preset privacy protection condition as the target face image.

[0176] In some embodiments, the preset privacy protection condition can include at least one of the following:

[0177] The face of the target user;

[0178] The face of a specific user other than the target user;

[0179] Or, the face of the target user and the face of a specific user other than the target user;

[0180] The specific user can be a user specified by the target user, or a user having an association relationship (e.g., an online / offline social friend relationship) with the target user, and the embodiments of the present application do not limit this.

[0181] For example, the specific event information to be published can be a mood written by user a on a social platform (e.g., a microblog, a public number article, etc.). The mood can include not only text but also life images uploaded by the user a. The life images include the face of the user a and the face of user b. User b can be anyone other than user a, such as a relative or friend of user a, a public figure, etc., and the embodiments of the present application do not limit this.

[0182] It can be seen that the target face image meeting the preset privacy protection condition is determined from the specific time information to be published of the target user by the business server automatically. Since the preset privacy protection condition is a pre-set filtering condition, the target face image can be automatically screened, so that the business server can quickly and targetedly generate the face adversarial sample for the target user automatically, without the target user actively triggering the process of adding the disturbance pattern, thereby ensuring the privacy security of the target user without the target user's awareness.

[0183] (2) The business server initiatively processes the historical published information of a single user regularly in batches

[0184] In some embodiments, the following steps can be included:

[0185] The business server determines a target user meeting a preset privacy protection condition, and acquires a historical image set published by the target user to a social platform or uploaded by the target user to a financial platform or a government affairs platform in a historical period. The historical image set includes at least one unprocessed candidate historical image of the target user.

[0186] The business server sets the candidate historical image meeting the preset privacy protection condition in the candidate historical image as the target face image.

[0187] It can be seen that the candidate historical image meeting the preset privacy protection condition is determined from the historical image set published by the target user in the historical period by the business server automatically. Since the preset privacy protection condition is a pre-set filtering condition, the target face image can be automatically screened in the first aspect, so that the business server can quickly and targetedly generate the face adversarial sample for the target user automatically, without the target user actively triggering the process of adding the disturbance pattern, thereby ensuring the privacy security of the target user without the target user's awareness.

[0188] In another aspect, even if the target user does not actively request the business server to perform privacy protection processing (e.g., generating a corresponding target adversarial sample) when publishing any image in the above historical image set to protect privacy security, or even if the business server does not automatically perform privacy protection processing (e.g., generating a corresponding target adversarial sample) when publishing any image in the above historical image set to protect the privacy security of the target user, or even if the business server does not periodically or irregularly perform privacy protection processing (e.g., generating a corresponding target adversarial sample) on the candidate historical images in the historical image set that meet the preset privacy protection condition after the target user publishes any image in the above historical image set for a period of time to protect the privacy security of the target user, the operation of performing privacy protection processing (e.g., generating a corresponding target adversarial sample) on the candidate historical images in the historical image set that meet the preset privacy protection condition after the scheme is adopted, that is, through leak checking and compensation, can improve the demand of the target user for privacy security.

[0189] (3) Business server processing scenario for real-time / historical publishing information of a single user based on user-initiated demand

[0190] In some embodiments, the following steps can be included:

[0191] The user terminal sends a second message to the business server, and the business server receives the second message from the user terminal. The second message indicates that at least one face image in specific event information to be published by the target user to a social platform or to be uploaded to a financial platform or a government platform is subjected to privacy protection processing.

[0192] The business server sets the face image in the specific event information that meets the preset privacy protection condition as the target face image according to the indication of the second message.

[0193] As can be seen, by actively requesting the business server to perform privacy protection processing (e.g., generating a corresponding target adversarial sample) when publishing any image in the above historical image set by the target user, the privacy security of the target user can be protected.

[0194] (4) Business server updates the mapping relationship between input and output

[0195] In some embodiments, the following steps can be included:

[0196] The business server determines the number of inputs of the target model, and inputs sub-images in the target face image to the target model as one input according to the number of inputs, to obtain the target adversarial sample, wherein each input corresponds to a pixel region in the target face image.

[0197] Alternatively, the business server determines the number of inputs to the target model, and inputs each target face image as an input to the target model according to the number of inputs, to obtain the target adversarial sample corresponding to each target face image.

[0198] Each input corresponds to a pixel region in the target face image.

[0199] As can be seen, by updating the mapping relationship between input and output, the business server can better adapt to image inputs containing different pixel content and perform targeted processing, thereby improving processing efficiency.

[0200] The above three methods for obtaining the target face image of the face to be generated adversarial example can be flexibly switched, configured, or combined, and any one or at least two of them can be implemented. This application embodiment does not limit this.

[0201] See Figure 5 , Figure 5 This paper illustrates a flowchart of a face image acquisition method according to an embodiment of this application. The following describes a face image acquisition method provided by this application embodiment, which is applied to a search engine. This application embodiment includes:

[0202] 501. The search engine receives the search command.

[0203] The search instructions include key information about the target user, such as the target user's name and nickname. The target user can be at least one user to be searched, and the search engine can simultaneously search for images related to at least two target users; however, this embodiment does not limit this.

[0204] 502. In response to the search command, the search engine sends an access request to the business server.

[0205] This access request is used to request a facial image that matches key information from the business server. For example, if the search instruction includes the name Zhang Wenwen, the access request could be to request a facial image tagged "Zhang Wenwen" from platforms such as Weibo and various public image libraries.

[0206] 503. The search engine obtains a set of candidate images that match the key information from the business server.

[0207] The candidate image set includes at least one target adversarial sample, and can also include at least one target face image (that is, a face image that is not subjected to privacy protection processing or a face adversarial sample that is not subjected to privacy protection processing by the technical solution of the embodiment of the application, which is not limited in the application). The target adversarial sample is obtained by inputting a target face image into a target model. The target model is obtained based on unsupervised learning. The similarity between the target adversarial sample and the target face image is lower than a preset threshold.

[0208] 504、The search engine determines a target image set from the candidate image set as the search result of the target user.

[0209] The target image set does not include a face adversarial sample. The target image set refers to at least one image published by a target user and including at least one face. The target image set can be derived from a face image of the target user published by user a or a face image of the target user published by user b, which is not limited in the embodiment of the application.

[0210] As can be seen, in the embodiment of the application, the business server side performs privacy protection processing on at least one image published by a user and including a face, so that the at least one image including a face generates a target adversarial sample. Therefore, when other search tools (for example, the search engine in the embodiment of the application) search and crawl these candidate image sets from the business server according to the search instruction including the key information, even if the search engine crawls the target adversarial sample containing the target user, the search engine cannot intelligently identify the target adversarial sample matching the key information of the target user when pre-processing (for example, screening) the candidate image set. That is, the target image set identified by the search engine from the candidate image set does not include the target adversarial sample. As can be seen, on the one hand, the embodiment of the application can interfere with face recognition by generating a target adversarial sample, so as to resist the search engine based on a neural network model from screening a face image (that is, the target adversarial sample) including the target user. On the other hand, the privacy and security of user data are also well protected.

[0211] Figures 1 to 6 Any technical feature mentioned in the embodiment of any one of the above Figures 7 to 11 corresponding embodiments also applies to the corresponding embodiments of the application, and the subsequent similar places will not be described.

[0212] The above describes a face image processing method in the embodiment of the application. The image processing device and the image search device for executing the face image processing method are described below.

[0213] Reference is made to Figure 7,like Figure 7 The schematic diagram of an image processing device 40 shown illustrates its ability to perform privacy protection processing on images to be published and previously published images, thereby generating adversarial examples of faces. This prevents search engines based on neural network models from directly identifying the target face image matching the specific user even after crawling face images of a specific user from the web (e.g., crawling face images previously published by the target user from a business server) and preprocessing the face images. The image processing device 40 in this embodiment can achieve the above-described... Figures 1-6 The steps in the face image processing method executed by the image processing device 40 in any corresponding embodiment. The functions implemented by the image processing device 40 can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The image processing device 40 may include an input / output module 401 and a processing module 402. The functional implementation of the input / output module 401 and the processing module 402 can be found in [reference needed]. Figures 1-6 The operations performed in any of the corresponding embodiments will not be described in detail here.

[0214] In some implementations, the input / output module 401 can be used to acquire a target face image to be processed;

[0215] The processing module 402 can be used to input the target face image into the target model to obtain the target adversarial sample, wherein the target model is obtained by unsupervised learning based on unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold.

[0216] The input / output module 401 is also used to publish the target adversarial sample obtained by the processing module 402, or the processing module updates the target face image with the target adversarial sample.

[0217] In some embodiments, the processing module 402 is specifically used for:

[0218] The target face image is subjected to nonlinear mapping processing in the convolutional layer to obtain a high-dimensional feature vector;

[0219] The high-dimensional feature vector is subjected to nonlinear mapping processing in the deconvolution layer to obtain the target feature vector;

[0220] The target feature vector is normalized and mapped to obtain the target adversarial sample.

[0221] In some embodiments, the processing module 402 is specifically used for:

[0222] The initial adversarial sample is subjected to augmentation processing to obtain the target adversarial sample, and the augmentation processing includes rotation and compression processing on the initial adversarial sample.

[0223] In some embodiments, the processing module 402 is further configured to:

[0224] The input / output module 401 obtains a training set and the face recognition model to be trained, and the training set includes a plurality of face images.

[0225] The training set is input into the face recognition model to obtain an adversarial sample set, and the adversarial sample set includes a plurality of adversarial samples.

[0226] Each adversarial sample in the adversarial sample set is compared with a corresponding face image in the training set, and if a loss value obtained by a loss function of the face recognition model is less than a preset threshold, the adversarial sample set is input into the face recognition model to train and update the face recognition model to obtain the target model.

[0227] In some embodiments, the loss function is obtained according to the following expression:

[0228]

[0229] wherein x is a face image in the training set, x1 is an adversarial sample in the adversarial sample set, represents a feature vector output by the face recognition model;

[0230] The preset threshold satisfies the following inequality:

[0231] ||x-x′|| ∞ ≤∈

[0232] wherein ∈ is a selected perturbation size.

[0233] In some embodiments, the processing module 402 is specifically configured to:

[0234] The input / output module 401 obtains an initial face image.

[0235] The initial face image is subjected to region division to obtain a plurality of sub-images.

[0236] Part or all of the plurality of sub-images are used as the target face image.

[0237] In some embodiments, the plurality of sub-images include at least one first region and at least one second region, wherein the first region is a non-face key feature region, and the second region is a face key feature region.

[0238] In some embodiments, the target face image is an initial face image obtained.

[0239] In some embodiments, the processing module 402 is specifically configured to:

[0240] receive a first message from a user terminal through the input and output module, the first message including specific event information to be published by a target user to a social platform, or to be uploaded to a financial platform or a government affairs platform, the specific event information including at least one initial face image;

[0241] set the initial face image in the specific event information that meets a preset privacy protection condition as the target face image.

[0242] In some embodiments, the processing module 402 is specifically configured to:

[0243] determine a target user that meets a preset privacy protection condition;

[0244] obtain a historical image set of the target user that has been published to a social platform, or has been uploaded to a financial platform or a government affairs platform within a historical period, the historical image set including at least one unprocessed candidate historical image of the target user;

[0245] set the candidate historical image in the candidate historical image that meets the preset privacy protection condition as the target face image.

[0246] In some embodiments, the target face image is derived from at least one of the following manners:

[0247] specific event information to be published by a target user on a social platform, the specific event information including at least one initial face image;

[0248] or, a historical image set published by a target user on a social platform within a historical period, the historical image set including at least one unprocessed candidate historical image of the target user, the candidate historical image including an image of the target user that meets a preset privacy protection condition.

[0249] In some embodiments, the target face image is derived from at least one of the following manners:

[0250] a historical image set to be uploaded by a target user on a financial platform or a government affairs platform;

[0251] or, a historical image set uploaded by a target user on a financial platform or a government affairs platform within a historical period;

[0252] wherein the historical image set includes at least one unprocessed candidate historical image of the target user.

[0253] In some embodiments, the processing module 402 is specifically configured to:

[0254] receive, by the input / output module 401, a second message from a user terminal, the second message indicating that at least one face image in specific event information to be published to a social platform or to be uploaded to a financial platform or a government platform by a target user is to be subjected to privacy protection processing;

[0255] According to the indication of the second message, the face image in the specific event information that meets the preset privacy protection condition is set as the target face image.

[0256] In some embodiments, the processing module 402 is specifically configured to:

[0257] determine the number of inputs of the target model;

[0258] According to the number of inputs, input each sub-image in the target face image to the target model as an input, to obtain the target adversarial sample, wherein each input corresponds to a pixel region in the target face image.

[0259] In the embodiments of the present application, since the target model is obtained based on unsupervised learning, the similarity between the image output by the target model (i.e., the target adversarial sample described above) and the target face image is lower than a preset threshold, that is, the target adversarial sample obtained after processing the target face image by the target model is different from the target face image before being processed by the target model. Therefore, on the one hand, when the target adversarial sample is published on a business server (such as a social platform), even if a search engine crawls the target adversarial sample, the target adversarial sample will be misidentified due to its adversarial nature, or the target adversarial sample cannot be bound to the identity information of the target user to be searched, so as to effectively protect the privacy data of the target user from being maliciously collected or utilized. On the other hand, after updating the target face image uploaded to a business server (such as a social platform, a financial platform, a government platform, or other platforms that need to upload privacy images including the face of a target user) to the target adversarial sample, even if the target adversarial sample uploaded to the business server is leaked due to some reasons, or is crawled by a search engine after being leaked, the target adversarial sample will be misidentified due to its adversarial nature, or the target adversarial sample cannot be bound to the identity information of the target user to be searched, so as to effectively protect the privacy data of the target user from being maliciously collected or utilized.

[0260] The image processing apparatus 40 performing the face image processing method and the image search apparatus 50 performing the face image acquisition method in the embodiments of the present application are respectively described from the perspective of the modular functional entities above, and are respectively described from the perspective of hardware processing below. It should be noted that the image processing apparatus 40 performing the face image processing method and the image search apparatus 50 performing the face image acquisition method in the embodiments of the present application can be respectively implemented by the hardware only, or can be respectively implemented by hardware plus software with the firmware. Figure 7 The entity device corresponding to the input / output module 401 in the embodiment shown can be an input / output unit, a transceiver, a radio frequency circuit, a communication module, an output interface, etc., and the entity device corresponding to the processing module 402 can be a processor. Figure 7 The image processing apparatus 40 shown can have a structure as shown in FIG. 4A, when Figure 9 The image processing apparatus 40 shown can have a structure as shown in FIG. 4A, when Figure 7 The image processing apparatus 40 shown can have a structure as shown in FIG. 4A, when Figure 9 The image processing apparatus 40 shown can have a structure as shown in FIG. 4A, when Figure 9 The processor and the transceiver in the image processing apparatus 40 can realize the same or similar functions of the input / output module 401 and the processing module 402 provided in the apparatus embodiment corresponding to the image processing apparatus 40, Figure 9 The memory in the image processing apparatus 40 stores the computer program required to be invoked by the processor when executing the face image processing method.

[0261] The image search apparatus 50 performing the face image acquisition method in the embodiments of the present application can have a structure as shown in FIG. 5A, when Figure 8 The entity device corresponding to the transceiving module 501 in the embodiment shown can be an input / output unit, a transceiver, a radio frequency circuit, a communication module, an output interface, etc., and the entity device corresponding to the processing module 502 can be a processor. Figure 8 The image search apparatus 50 shown can have a structure as shown in FIG. 5A, when Figure 9 The image search apparatus 50 shown can have a structure as shown in FIG. 5A, when Figure 8 The image search apparatus 50 shown can have a structure as shown in FIG. 5A, when Figure 9 The image search apparatus 50 shown can have a structure as shown in FIG. 5A, when Figure 9 The processor and the transceiver in the image search apparatus 50 can realize the same or similar functions of the transceiving module 501 and the processing module 502 provided in the apparatus embodiment corresponding to the image search apparatus 40, Figure 9 The memory in the image search apparatus 50 stores the computer program required to be invoked by the processor when executing the face image acquisition method.

[0262] The present application also provides another image processing apparatus, which can have a structure as shown in FIG. 6A, when Figure 10As shown, for the convenience of illustration, only part of the embodiment of the present application is shown, and the specific technical details are not disclosed, please refer to the embodiment method part of the present application. The image processing device can be any image processing device including a mobile phone, a tablet computer, a personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA), a sales image processing device (English full name: Point of Sales, English abbreviation: POS), a vehicle-mounted computer, etc. Take the mobile phone as an example:

[0263] Figure 10 The figure shows a block diagram of part of the structure of the mobile phone related to the image processing device provided by the embodiment of the present application. Referring to Figure 10 , the mobile phone includes: radio frequency (English full name: Radio Frequency, English abbreviation: RF) circuit 710, memory 720, input unit 730, display unit 740, sensor 780, audio circuit 760, wireless fidelity (English full name: wireless-fidelity, English abbreviation: Wi-Fi) module 7100, processor 780, and power supply 790, etc. Those skilled in the art can understand, Figure 7 The structure of the mobile phone shown in the figure does not constitute a limitation on the mobile phone, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0264] The specific components of the mobile phone will be described below Figure 10

[0265] ​The RF circuit 710 can be used for receiving and sending signals in the process of information or communication, in particular, receiving the downlink information from the base station and processing by the processor 780; in addition, sending the uplink data to the base station. Generally, the RF circuit 710 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 710 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0266] The memory 720 can be used to store software programs and modules, and the processor 780 can execute various function applications and data processing of the mobile phone by running the software programs and modules stored in the memory 720. The memory 720 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 720 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0267] The input unit 730 can be used to receive input digital or character information, and to generate key signal input with respect to user setting of the mobile phone and control of the function. Specifically, the input unit 730 can include a touch panel 731 and other input device 732. The touch panel 731, also called a touch screen, can collect a touch operation (such as an operation of a user using a finger, a stylus, or any suitable object or accessory on or near the touch panel 731) of the user on or near it, and drive the corresponding connection device according to the pre-set program. Optionally, the touch panel 731 can include two parts of a touch detection device and a touch controller. Among them, the touch detection device detects the touch position of the user and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, and converts it into touch coordinates and sends it to the processor 780, and can receive the command from the processor 780 and execute it. In addition, the touch panel 731 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 731, the input unit 730 can also include other input device 732. Specifically, the other input device 732 can include one or more of a physical keyboard, a function key (such as a volume control button, an on-off button, etc.), a trackball, a mouse, a joystick, etc.

[0268] The display unit 740 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 740 can include a display panel 741, which can be configured in the form of a liquid crystal display (English full name: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (English full name: Organic Light-Emitting Diode, English abbreviation: OLED), etc. Further, the touch panel 731 can cover the display panel 741, and when the touch panel 731 detects a touch operation on or near it, it is transmitted to the processor 780 to determine the type of touch event, and then the processor 780 provides corresponding visual output on the display panel 741 according to the type of touch event. Although in the Figure 7 , the touch panel 731 and the display panel 741 are realized as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 731 and the display panel 741 can be integrated to realize the input and output functions of the mobile phone.

[0269] The phone can also include at least one sensor 780, such as an optical sensor, a motion sensor, and other sensors. Specifically, the optical sensor can include an ambient light sensor to adjust the brightness of the display panel 741 according to the brightness of ambient light, and a proximity sensor to turn off the display panel 741 and / or the backlight when the phone is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (usually three axes), and when at rest, it can detect the magnitude and direction of gravity, which can be used for applications that identify the phone posture (such as switching between landscape and portrait screens, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometers, taps), and the like. As for other sensors that the phone can also be configured, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, they will not be described here.

[0270] The audio circuit 760, the speaker 761, and the microphone 762 can provide an audio interface between the user and the phone. The audio circuit 760 can convert the received audio data into an electrical signal, transmit it to the speaker 761, and convert it into a sound signal output by the speaker 761; on the other hand, the microphone 762 converts the collected sound signal into an electrical signal, which is received by the audio circuit 760 and converted into audio data, which is then output to the processor 780 for processing, and then transmitted to another phone via the RF circuit 710, or output to the memory 720 for further processing.

[0271] Wi-Fi belongs to a short-range wireless transmission technology. The phone can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 7100, which provides users with wireless broadband Internet access. Although Figure 9 Although the Wi-Fi module 7100 is shown, it is understood that it does not belong to the necessary components of the phone, and can be omitted as needed without changing the essence of the application.

[0272] The processor 780 is the control center of the phone, which connects all parts of the phone through various interfaces and lines, executes various functions of the phone and processes data by running or executing software programs and / or modules stored in the memory 720, and calling data stored in the memory 720, thereby monitoring the whole phone. Optionally, the processor 780 can include one or more processing units; preferably, the processor 780 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication. It is understood that the above-mentioned modem processor can also not be integrated into the processor 780.

[0273] The mobile phone further includes a power supply 790 (such as a battery) for supplying power to various components, which can be logically connected to the processor 780 through a power management system, so as to realize functions such as charge management, discharge management, and power consumption management through the power management system.

[0274] Although not shown, the mobile phone can further include a camera, a Bluetooth module, and the like, which will not be described herein.

[0275] In the embodiments of the present application, the processor 780 included in the mobile phone further has control over the method flow executed by the image processing apparatus 40 shown in the above. Figure 10 The steps executed by the image processing apparatus in the above embodiments can be based on the structure of the mobile phone shown in the above. Figure 10 For example, the processor 722 executes the following operations by invoking instructions in the memory 732:

[0276] The target face image to be processed is acquired through the input unit 730;

[0277] The target face image acquired by the input unit 730 is input into a target model to obtain a target adversarial sample, wherein the target model is obtained based on unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold.

[0278] The target adversarial sample obtained by the processing module is published through the input unit 730, or the target face image is updated to the target adversarial sample by the processing module.

[0279] In some other embodiments of the present application, the processor 780 included in the mobile phone further has control over the method flow executed by the image search apparatus 50 shown in the above. Figure 8 The steps executed by the image processing apparatus in the above embodiments can be based on the structure of the mobile phone shown in the above. Figure 10 For example, the processor 722 executes the following operations by invoking instructions in the memory 732:

[0280] A search instruction is received through the input unit 730, and the search instruction includes key information of a target user;

[0281] In response to the search instruction, a candidate image set matching the key information is acquired from a service server through the transceiver module; wherein the candidate image set includes at least one target adversarial sample, the target adversarial sample is obtained by inputting a target face image into a target model; the target model is obtained based on unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold.

[0282] Determine a target image set as a search result of the target user from the candidate image set, and the target image set does not include the target adversarial sample.

[0283] The embodiments of the present application further provide an image processing apparatus for implementing the above-mentioned face image processing method, or a search apparatus for implementing the above-mentioned face image acquisition method, as shown in Figure 11 Figure 11 Fig. 10 is a schematic diagram of a server structure according to an embodiment of the present application. The server 1020 can have great differences due to different configurations or performances, and can include one or more central processing units (CPU) 1022 (for example, one or more processors) and a memory 1032, one or more storage media 1030 (for example, one or more mass storage devices) for storing application programs 1042 or data 1044. The memory 1032 and the storage media 1030 can be temporary storage or persistent storage. The programs stored in the storage media 1030 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the server. Further, the central processing unit 1022 can be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1030 on the server 1020.

[0284] The server 1020 can further include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, and / or one or more operating systems 1041, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0285] The steps performed by the service server (for example, the image processing apparatus 40 shown in Figure 7 ) or the search engine (for example, the search apparatus 50 shown in Figure 8 ) in the above-mentioned embodiments can be based on the structure of the server 1020 shown in Figure 11 . For example, the steps performed by the image processing apparatus 40 shown in Figure 7 in the above-mentioned embodiments can be based on the server structure shown in Figure 11 . For example, the processor 1022 can execute the following operations by invoking the instructions in the memory 1032:

[0286] Obtain the target face image to be processed through the input / output interface 1058;

[0287] ​input the target face image obtained by the input / output interface 1058 to a target model to obtain a target adversarial sample, where the target model is obtained based on unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold;

[0288] The target adversarial sample obtained by the processing module is published through the input / output interface 1058, or the target face image is updated to the target adversarial sample by the processing module.

[0289] For example, the steps performed by the search device 50 shown in the above embodiments can be based on the server structure shown in the above embodiments. For example, the processor 1022 performs the following operations by calling the instructions in the memory 1032: Figure 8 Figure 11 For example, the processor 1022 performs the following operations by calling the instructions in the memory 1032:

[0290] The search instruction is received through the input / output interface 1058, and the search instruction includes key information of a target user;

[0291] In response to the search instruction, a candidate image set matching the key information is obtained from a service server through the transceiver module, where the candidate image set includes at least one target adversarial sample, the target adversarial sample is obtained by inputting a target face image to a target model, the target model is obtained based on unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold;

[0292] The target image set is determined from the candidate image set as the search result of the target user, and the target image set does not include a target adversarial sample.

[0293] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0294] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and module described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0295] ​In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the embodiments of the device described above are merely schematic; for example, the division of the modules is merely a logical function division; an actual implementation can be another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different modules can be indirect couplings or communication connections through some interfaces, devices or modules, and can be electrical, mechanical or in other forms.

[0296] The modules described as separated components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e., can be located in one place, or can be distributed on a plurality of network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0297] In addition, each functional module in each of the embodiments of the present application can be integrated in one processing module, or each module can be physically present alone, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can be stored in a computer readable storage medium.

[0298] In the above embodiments, all or part can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part can be realized in the form of a computer program product.

[0299] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on the computer, the flow or function described in the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can store or be integrated into a data storage device such as a server, data center, etc. containing one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0300] The above describes the technical solutions provided by the embodiments of the present application in detail. The principles and implementation manners of the embodiments of the present application are described by applying specific examples. The above examples are only used to help understand the method and core idea of the embodiments of the present application; at the same time, for those skilled in the art, according to the idea of the embodiments of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as a limitation of the embodiments of the present application.

Claims

1. A face image processing method, characterized by, The method is executed by a business server, and the method comprises: obtaining a target face image to be processed; inputting the target face image into a target model to obtain a target adversarial sample, wherein the target model is obtained based on unsupervised learning, and the similarity between the target adversarial sample and the target face image is lower than a preset threshold; it comprises: performing nonlinear mapping processing on the target face image in a convolution layer to obtain a high-dimensional feature vector; performing nonlinear mapping processing on the high-dimensional feature vector in a deconvolution layer to obtain a target feature vector; performing normalization mapping processing on the target feature vector to obtain the target adversarial sample; publishing the target adversarial sample, or updating the target face image to the target adversarial sample; wherein, before inputting the target face image into the target model, the method further comprises: obtaining a training set and a face recognition model to be trained, the training set comprising a plurality of face images; inputting the training set into the face recognition model to obtain an adversarial sample set, the adversarial sample set comprising a plurality of adversarial samples; comparing each adversarial sample in the adversarial sample set with the corresponding face image in the training set, if the loss value obtained by the loss function of the face recognition model is less than a preset threshold, inputting the adversarial sample set into the face recognition model to train and update the face recognition model, and obtaining the target model.

2. The method of claim 1, wherein, The loss function is obtained according to the following expression: wherein x is a face image in the training set, x' is an adversarial sample in the adversarial sample set, represents a feature vector output by the face recognition model; The preset threshold satisfies the following inequality: wherein, is the selected perturbation size.

3. The method according to claim 1 or 2, characterized in that, The obtaining of the target face image to be processed comprises: obtaining an initial face image; dividing the initial face image into regions to obtain a plurality of sub-images; using part or all of the sub-images as the target face image.

4. The method according to claim 1 or 2, characterized in that, The obtaining of the target face image to be processed comprises: receiving a first message from a user terminal, the first message comprising specific event information to be published to a social platform or uploaded to a financial platform or a government platform by a target user, the specific event information comprising at least one initial face image; setting the initial face image in the specific event information that meets a preset privacy protection condition as the target face image.

5. The method according to claim 1 or 2, characterized in that, The obtaining of the target face image to be processed comprises: determining a target user who meets a preset privacy protection condition; obtaining a historical image set of the target user that has been published to a social platform or uploaded to a financial platform or a government platform in a historical period, the historical image set comprising at least one unprocessed candidate historical image of the target user; setting the candidate historical image in the candidate historical image set that meets the preset privacy protection condition as the target face image.

6. The method according to claim 1 or 2, characterized in that, The obtaining of the target face image to be processed comprises: receiving a second message from a user terminal, the second message indicating that at least one face image in specific event information to be published to a social platform or uploaded to a financial platform or a government platform by a target user is to be subjected to privacy protection processing; setting the face image in the specific event information that meets a preset privacy protection condition as the target face image according to the indication of the second message.

7. An image processing apparatus characterized by comprising: The image processing device comprises: at least one processor and a memory; wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program stored in the memory to perform the method of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, a computer program product, comprising instructions which, when executed on a computer, cause the computer to carry out the method of any one of claims 1-6.

Images