A face image processing method and device and a storage medium

Abstract

Embodiments of the present application relate to the technical field of computer vision, and provide a face image processing method and device and a storage medium, the method comprising: obtaining candidate face feature coefficients of a first user; obtaining a candidate face shape and a candidate face texture according to the candidate face feature coefficients, a preset eigenface, an average face shape and an average face texture vector; generating a candidate adversarial patch based on the candidate face shape and the candidate face texture; obtaining a candidate adversarial sample, the candidate adversarial sample being obtained by superimposing the candidate adversarial patch and an attacker face image of a second user; obtaining a similarity between the candidate adversarial sample and a target face image; if the similarity is less than a preset value, updating the candidate feature coefficients and the candidate adversarial patch until the similarity is greater than the preset value, and outputting the candidate adversarial patch at this time as a target adversarial patch. The present scheme can improve the adversarial property of the adversarial patch and avoid reducing the attack effect due to deformation or face fitting degree.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer vision, and particularly relate to a face image processing method and device and a storage medium. BACKGROUND

[0002] Face recognition algorithms are widely used in payment, security and identity verification and many other fields. However, the current face recognition model is easy to be deceived by malicious adversarial samples constructed, and presents a certain vulnerability. These malicious adversarial samples bring certain security risks to the application of face recognition. At present, effective adversarial samples are mainly generated based on adversarial attack technology, and the security of the face recognition algorithm is evaluated based on the effective adversarial samples. Since the face recognition system usually collects face images in the real world and performs recognition, the research of physical world attack technology is more important, such as constructing some typical adversarial eyes, masks and the like.

[0003] At present, the adversarial attack form of the adversarial hat is mainly adopted, that is, the adversarial noise is limited within the range of the hat, and the two-dimensional curvature approximation method is used to simulate the change of the hat in the real world, so that the face recognition model can be successfully attacked in the physical world.

[0004] In the research and practice process of the prior art, the inventors of the embodiments of the present application found that only by simulating the deformation of the real world through curvature approximation, the real 3D shape information is not used for adversarial sample generation, so that the attack effect is limited. SUMMARY

[0005] Embodiments of the present application provide a face image processing method, device and storage medium, which can improve the adversarial property of the adversarial patch, avoid reducing the attack effect due to deformation or facial adhesion, and also solve the problem of deformation of the manufactured physical adversarial patch, thereby ensuring the attack effect and improving the success rate of attacking the live detection module in the face recognition system.

[0006] In a first aspect, the embodiments of the present application provide a face image processing method, which comprises:

[0007] obtaining candidate face feature coefficients of a first user;

[0008] obtaining candidate face shapes and candidate face textures according to the candidate face feature coefficients, a preset eigenface, an average face shape and an average face texture vector;

[0009] generating a candidate adversarial patch based on the candidate face shapes and the candidate face textures;

[0010] obtaining a candidate adversarial sample, the candidate adversarial sample being obtained by superimposing the candidate adversarial patch and an attacker face image of a second user.

[0011] obtaining a similarity between the candidate adversarial sample and the target face image of the first user;

[0012] If the similarity is less than a preset value, updating the candidate feature coefficient and the candidate adversarial patch until the similarity between the candidate adversarial sample and the target face image is greater than the preset value, and outputting the candidate adversarial patch when the similarity is greater than the preset distance as the target adversarial patch.

[0013] In a second aspect, an embodiment of the present application provides an image processing apparatus having a function of implementing the face image processing method according to the first aspect. The function can be implemented by hardware, or by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above functions, which can be software and / or hardware.

[0014] In some embodiments, the image processing apparatus includes:

[0015] an input / output module configured to obtain a candidate face feature coefficient of a first user;

[0016] a processing module configured to obtain a candidate face shape and a candidate face texture according to the candidate face feature coefficient, a preset feature face, an average face shape, and an average face texture vector obtained by the input / output module;

[0017] The processing module is further configured to generate a candidate adversarial patch based on the candidate face shape and the candidate face texture, obtain a candidate adversarial sample by superimposing the candidate adversarial patch on an attacker face image of a second user, and obtain a similarity between the candidate adversarial sample and a target face image of the first user.

[0018] If the similarity is less than a preset value, the processing module is further configured to update the candidate feature coefficient and the candidate adversarial patch until the similarity between the candidate adversarial sample and the target face image is greater than the preset value, and output the candidate adversarial patch when the similarity is greater than the preset distance as the target adversarial patch through the input / output module.

[0019] In a possible design, the processing module is specifically configured to:

[0020] determine a preset topological connection of a face three-dimensional patch;

[0021] perform projection transformation on the preset topological connection to obtain a binary mask, the binary mask being used to indicate a position where the candidate adversarial patch is to be added in the attacker face image;

[0022] add the candidate adversarial patch to the attacker face image according to the position indicated by the binary mask to obtain the candidate adversarial sample.

[0023] In a possible design, the processing module is specifically configured to:

[0024] obtain a target face image;

[0025] input the target face image into a three-dimensional morphable model, and calculate candidate face feature coefficients according to a preset average face shape, a preset average face texture vector, a preset feature face, and the target image in the three-dimensional morphable model;

[0026] obtain a candidate face shape according to the average face shape, the preset feature face, and the candidate feature coefficients, and obtain a candidate face texture according to the preset feature face and the candidate feature coefficients.

[0027] In some embodiments, the attacker face image is a first face image of the second user from the target image set; and the processing module is specifically configured to:

[0028] obtain the candidate adversarial patch and the attacker face image;

[0029] superimpose the candidate adversarial patch and the first face image to obtain a 3D face adversarial sample;

[0030] map the 3D face adversarial sample to a 2D candidate adversarial sample.

[0031] In a possible design, the processing module is specifically configured to:

[0032] obtain a preset topological connection of a face three-dimensional patch, a lighting parameter value, and a shooting angle;

[0033] input the 3D face adversarial sample into a renderer according to the preset topological connection, the lighting parameter value, and the shooting angle to obtain the candidate adversarial sample.

[0034] In a possible design, the processing module is specifically configured to:

[0035] obtain a second face image collected by a shooting device in real time through the input and output module, the second face image being an image collected after a second user wears a physical 3D adversarial patch;

[0036] use the second face image as the candidate adversarial sample.

[0037] In a possible design, the face image processing method is implemented based on a wearable device with a flexible screen; the flexible screen includes N curved sub-blocks; after the input / output module outputs the target adversarial patch, the processing module is further configured to:

[0038] receive, by the input / output module, a first instruction, the first instruction including one of an instruction sent by a control end, an input operation of a user on the wearable device, and a recorded voice of the user;

[0039] in response to the first instruction, generate a 3D face adversarial pattern and generate a second instruction, the second instruction including a position sequence of face features in each sub-block on the flexible screen;

[0040] in response to the second instruction, control each sub-block to respectively display each pixel point of the 3D face adversarial pattern according to the position sequence in the second instruction.

[0041] In a third aspect, an embodiment of the present application provides a computer device, including at least one connected processor, a memory, and a transceiver, wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program in the memory to execute the method provided in the first aspect and various possible designs in the first aspect.

[0042] In yet another aspect, an embodiment of the present application provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method provided in the first aspect and various possible designs in the first aspect.

[0043] In yet another aspect, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the first aspect and various possible designs in the first aspect.

[0044] Compared with the prior art, in the scheme provided by the embodiments of the present application, the candidate face shape and the candidate face texture are obtained according to the candidate face feature coefficient, the preset eigenface, the average face shape and the average face texture vector, and the candidate face feature coefficient is the personalized feature coefficient of the face of the first user as the attacked person based on the preset eigenface (a given fixed value), so the candidate face feature only represents the facial features of the first user. Therefore, in one aspect, the candidate face shape and the candidate face texture obtained based on the candidate face feature coefficient with such features can represent the key features of the original face and fit the real face. Therefore, the candidate adversarial patch generated based on the candidate face shape and the candidate face texture has very good adversarial property, and compared with the 2D adversarial patch, the attack effect is not reduced due to deformation or facial fitting degree; in another aspect, when the face verification task is performed based on the candidate adversarial patch, the attack effect of the candidate adversarial patch on the face recognition system can be effectively verified, and the embodiments of the present application determine the target adversarial patch through limited cycles, so that the finally obtained target adversarial patch fully considers the 3D transformation in the real scene, and therefore can better fit the real face. Compared with the 2D adversarial patch in the prior art, the problem of deformation of the physical adversarial patch is solved, so that the attack effect is guaranteed, and the success rate of the live detection module in attacking the face recognition system is improved. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 A face recognition scene schematic diagram provided by the embodiments of the present application;

[0046] Figure 2 A flowchart of the face image processing method in the embodiments of the present application;

[0047] Figure 3 A flowchart of the initial face image, the 3D adversarial patch to the 2D face image in the embodiments of the present application;

[0048] Figure 4a A schematic diagram of the 3D adversarial patch in the embodiments of the present application;

[0049] Figure 4b A flowchart of the process from the target adversarial patch to the face verification task for testing in the embodiments of the present application;

[0050] Figure 5 A structural schematic diagram of the image processing device in the embodiments of the present application;

[0051] Figure 6 A structural schematic diagram of the physical device for implementing the face image processing method in the embodiments of the present application;

[0052] Figure 7 Fig. 1 is a schematic diagram of a structure of a server for implementing a face image processing method in an embodiment of the present application;

[0053] Figure 8 Fig. 2 is a schematic diagram of a structure of a mobile phone for implementing a face image processing method in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects (for example, the first user and the second user in the embodiments of the present application represent users of different identities, respectively) and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or modules does not have to be limited to only those steps or modules clearly listed, but can include other steps or modules that are not clearly listed or inherent to the process, method, product, or device. The division of modules appearing in the embodiments of the present application is only a logical division, and in actual application, another division manner can be used, for example, a plurality of modules can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be through an interface, and the indirect coupling or communication connection between the modules can be electrical or other similar forms, which are not limited in the embodiments of the present application. In addition, the modules or sub-modules described as separate components can or can not be physically separated, can or can not be physical modules, or can be distributed into a plurality of circuit modules, and some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0055] The embodiments of the present application provide a face image processing method, device, and storage medium, which can be applied to robust detection of a face recognition system, for example, in a smart security, smart monitoring, or smart education scenario, and the face recognition result can be used to determine user identity information. The scheme can be applied to a server or a service terminal. The embodiments of the present application take the server as an example, and when applied to the service terminal side, the embodiments of the server can be referred to, and details are not described herein.

[0056] In some embodiments, the embodiments of the present application can be applied to a face recognition scenario as shown in Figure 1 Figure 1 ​The face recognition system includes at least one face recognition device (which can also be understood as a face recognition model, and the embodiments of the present application do not distinguish between the two), at least one image acquisition device (which can also be referred to as a shooting device, and the embodiments of the present application do not distinguish between the two, Figure 1 The image processing device and the face recognition device can interact with each other, and the image acquisition device and the face recognition device can interact with each other. The face recognition device and the image acquisition device can be centrally deployed or separately deployed, and the embodiments of the present application do not limit this, but only take the separately deployed as an example.

[0057] The image processing device is configured to generate a 3D adversarial patch.

[0058] The 3D adversarial patch is configured to print a 3D adversarial patch entity according to the printing instruction received from the image processing device.

[0059] The shooting device is configured to acquire a face image of a user wearing the 3D adversarial patch and transmit the face image to the face recognition device for a face verification task.

[0060] The face recognition device is configured to verify the identity of the face image acquired by the shooting device.

[0061] When the face image processing method is implemented in the application scenario as shown in Figure 1 When the face image processing method is implemented in the application scenario as shown in

[0062] The scheme of the embodiments of the present application can be implemented based on artificial intelligence technology, specifically, it relates to the field of computer vision technology in artificial intelligence technology and the fields of cloud computing, cloud storage and database in cloud technology, which will be introduced respectively.

[0063] Artificial Intelligence (AI) is the theory, method, technology and application system that use digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0064] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other several major directions.

[0065] Computer Vision (CV) Computer vision is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further process graphics so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, trying to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies, as well as common face recognition, fingerprint recognition and other biometric identification technologies.

[0066] It should be particularly pointed out that the server and the service terminal (for example, the face recognition device) involved in the embodiments of the present application can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and basic cloud computing services such as big data and artificial intelligence platforms. The adversarial sample generation device involved in the embodiments of the present application can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a personal digital assistant, and the like, but is not limited thereto. The face recognition device and the server can be directly or indirectly connected through wired or wireless communication, and the embodiments of the present application do not limit this.

[0067] In view of the problem that the adversarial hat does not consider the 3D transformation in the real scene, the adversarial patch is deformed when being attached to the face, and the effect is poor, the embodiments of the present application mainly provide the following technical solutions:

[0068] The 3D adversarial mask combined with the adversarial technology has depth information and texture information closer to a real face. Therefore, in an actual face recognition system, the 3D adversarial mask can not only obtain a high similarity with a real victim face, but also can effectively escape from a live detection module. Therefore, the attack technology based on the 3D adversarial mask of the adversarial technology can better evaluate the robustness of the current face recognition system.

[0069] In the following, the technical solutions of the embodiments of the present application will be described Figure 2 by way of example with reference to

[0070] With reference to Figure 2 , a face image processing method provided by the embodiments of the present application will be introduced below, and the face recognition system includes a given face library, and the face library includes face images of multiple users. The embodiments of the present application include:

[0071] 201, obtaining candidate face feature coefficients of a first user.

[0072] The first user is an attacked user, and is a legal user in a face library (for example, a given face library ) of a face recognition model.

[0073] The candidate face feature coefficients are face feature coefficients of a face (that is, a specific user face) of the attacked user. It is assumed that B id ,B exp ,B t The basis is a given face library B id ,B exp ,B tThe orthogonal bases respectively represent the individual, expression and texture of the face, and the candidate face feature coefficients include α, β, δ, and α, β, δ are the coefficients of the individual, expression and texture of the face of a specific user (for example, the face of the first user) obtained under the given orthogonal bases. If the orthogonal bases of the subspace are arranged according to the image array, it can be seen that B id ,B exp ,B t presents the shape of the face, and thus B id ,B exp ,B t Also known as Eigenface, the embodiments of the present application do not distinguish between the two.

[0074] In some embodiments, in order to make the finally produced 3D adversarial patch have depth information and texture information closer to real faces, the candidate face feature coefficients of the first user can be obtained by the following operations:

[0075] Obtain a target face image, which is a face image of the first user;

[0076] Input the target face image into a three-dimensional morphable model, and calculate the candidate face feature coefficients according to the preset average face shape, average face texture vector, preset Eigenface in the three-dimensional morphable model, and the target face image.

[0077] The three-dimensional morphable model is a digital face model, and the embodiments of the present application mainly refer to (3D Morphable models, 3DMM). The 3DMM is established on the basis of a three-dimensional face database, and is statistically constrained by face shape and face texture, while considering the influence of face posture and illumination factors, so that the generated three-dimensional face model has high precision. The 3DMM uses a statistical method to construct a 3D model of a 2D face to express the face morphable model. In the embodiments of the present application, the three-dimensional morphable model can use a large-scale 3D morphable model (Large scale 3D Morphable Models, LSFM), a Nonlinear-3DMM or other model structures. The embodiments of the present application do not limit the model structures used, as long as they can realistically restore real-world faces.

[0078] 202. Obtain a candidate face shape and a candidate face texture according to the candidate face feature coefficients, the preset Eigenface, the average face shape and the average face texture vector.

[0079] Correspondingly, in some embodiments, after obtaining the candidate face feature coefficient of the first user based on the above operation, the candidate face shape can be obtained according to the average face shape, the preset feature face and the candidate feature face coefficient, and the candidate face texture can be obtained according to the preset feature face and the candidate feature face coefficient.

[0080] For example, given a face picture For a given face library, the 3D face modeling method is used to reconstruct the corresponding three-dimensional face information of x, and then a 3D mask can be obtained through 3D printing technology. Specifically, first, the shape S and texture T of the face x are reconstructed based on the three-dimensional deformable face model (3DMM), which is specifically represented as follows:

[0081]

[0082] Among them, is the average face shape in the given face library , is the average face texture vector in the given face library , B id ,B exp ,B t are all obtained based on the given face library , respectively representing the orthogonal basis of the individual, expression and texture of the face. Alpha is the coefficient of the individual of a specific face under the given orthogonal basis, beta is the coefficient of the expression of a specific face under the given orthogonal basis, and delta is the coefficient of the texture of a specific face under the given orthogonal basis.

[0083] Then, based on the shape s and texture T of the face x reconstructed by the above formula (1) and formula (2), a 3D face image, i.e. a 3D adversarial patch, is obtained to provide data support for the face authentication task in each round of iteration process in the subsequent embodiments.

[0084] 203. Generate a candidate adversarial patch based on the candidate face shape and the candidate face texture.

[0085] The candidate adversarial patch can be used to generate a 3D face adversarial sample, for example Figure 4a as shown.

[0086] 204. Obtain a candidate adversarial sample.

[0087] Among them, the candidate adversarial sample is obtained by superimposing the candidate adversarial patch and the attacker face image of the second user.

[0088] In some embodiments, the candidate adversarial sample can be obtained by the following operation:

[0089] determining a preset topological connection of a three-dimensional face patch of a human face;

[0090] performing a projection transformation on the preset topological connection to obtain a binary mask, the binary mask being used to indicate a position at which the candidate adversarial patch is to be added to an attacker face image;

[0091] adding the candidate adversarial patch to the attacker face image according to the position indicated by the binary mask to obtain a candidate face adversarial sample.

[0092] It should be noted that the process of obtaining the candidate adversarial sample is a cyclic iteration process. The embodiments of the present application only take a single cycle as an example to illustrate the obtaining method of the candidate adversarial sample. Through multiple iterations, a candidate adversarial perturbation that achieves the expected attack effect and a face adversarial sample (i.e., a 3D face adversarial sample in subsequent embodiments) generated by adding the candidate adversarial perturbation can be obtained. The other iteration times are not described in detail herein.

[0093] In order to verify whether the candidate adversarial sample or the target adversarial sample achieves a better attack effect, the face recognition verification task needs to be performed on the generated candidate adversarial sample or target adversarial sample. The embodiments of the present application mainly perform the following methods one and two:

[0094] Method one: simulating a face verification task

[0095] When the attacker face image is a first face image of the second user from the target image set, the candidate adversarial sample can be obtained through the following steps a-c:

[0096] a. Obtaining the candidate adversarial patch and the attacker face image.

[0097] That is, in each iteration, a candidate adversarial patch is obtained as an adversarial perturbation to be added to the first face image to verify the attack effect of the candidate adversarial patch in this iteration.

[0098] b. Superimposing the candidate adversarial patch and the first face image to obtain a 3D face adversarial sample.

[0099] c. Mapping the 3D face adversarial sample to a 2D candidate adversarial sample.

[0100] The 2D candidate adversarial sample is used for subsequent face verification tasks to verify the attack effect of the candidate adversarial patch.

[0101] In some embodiments, the mapping of the 3D face adversarial sample to the 2D target adversarial sample comprises:

[0102] obtaining a preset topological connection of the face three-dimensional patches according to the preset face three-dimensional model, the preset face three-dimensional model, and the preset face three-dimensional model;

[0103] According to the light parameter value, the shooting angle, and the preset topological connection, the 3D face adversarial sample is input into a renderer to obtain the candidate adversarial sample.

[0104] Specifically, the embodiment of the present application can use a renderer to render the 3D face adversarial sample into a 2D target adversarial sample. The target adversarial sample obtained after R rendering can be represented by the following expression:

[0105] x' = R (a, b, q, g, p; tri) formula (3)

[0106] Wherein, R represents a renderer, indicating the process of rendering a 3D face into a 2D face picture, g represents light, p represents angle; Tri represents the topological connection mode of the face three-dimensional patches; F represents a face recognition model.

[0107] Correspondingly, after rendering the 3D candidate adversarial sample in each iteration into a 2D face image, the rendered 2D face picture can be used for subsequent face verification tasks to verify the attack effect of the candidate adversarial patch in this iteration.

[0108] It should be noted that in each iteration, a candidate adversarial patch is obtained, and after superimposing it on the first face image to obtain the 3D adversarial sample of the current iteration, in order to verify its attack effect, the 3D adversarial sample obtained in this iteration needs to be mapped into a 2D face image to realize the real-time face verification task of the current iteration based on the 2D face image, so as to verify whether the 3D adversarial patch obtained in this iteration can be output as a target adversarial patch with better attack effect.

[0109] It can be seen that after reconstructing the depth information and facial texture information of the face, the expected attack effect of the 3D adversarial patch is simulated in the digital world in each round of 3D adversarial patch generation process, so that the attack effect of the 3D adversarial patch obtained in each round can be verified in real time after being made into an entity in the physical world. Reach the expected, so it can be iterated and corrected constantly, and finally the target 3D adversarial patch with the expected attack effect can be output in a short time, which can reduce the adversarial loss between the digital world and the physical world on the one hand, and save the test cost and improve the test efficiency on the other hand.

[0110] Method two: real-time evaluation in the physical world

[0111] Specifically, in the actual test of the face verification task, the second user uses the identity of the first user to attack the face recognition system for testing. When the second user wears the 3D adversarial patch of the entity, the photographing device connected with the face recognition system or built-in the face recognition system can collect the face image of the upper body of the second user wearing the 3D adversarial patch and transmit it to the image processing device in the face recognition system, such as Figure 4b As shown in a flowchart for testing the face verification task from the target adversarial patch to the face recognition system, the image processing device obtains the candidate adversarial sample through the following steps:

[0112] Obtain the second face image collected by the photographing device in real time. The second face image can be a printed 3D adversarial patch entity or an image collected after the second user wears the 3D adversarial patch entity.

[0113] The second face image is used as the candidate adversarial sample.

[0114] It can be seen that through this way, the face adversarial image in the physical world (i.e., the second face image with adversarial attack effect) can be directly collected, and then rendered into a 2D face image, so that it can be used to attack the face recognition device in the face recognition system in real time. Through the collected second face image in the physical world, the success rate of the 3D adversarial patch attack on the physical world can be verified more effectively, so as to reduce the adversarial loss between the digital world and the physical world, and optimize the generation of the 3D adversarial patch, so as to optimize the 3D adversarial patch in real time to make a 3D adversarial patch with better attack effect.

[0115] 205, Obtain the similarity between the candidate adversarial sample and the target face image of the first user.

[0116] 206, If the similarity is less than a preset value, update the candidate feature coefficient and the candidate adversarial patch until the similarity between the candidate adversarial sample and the target face image is greater than the preset value. The candidate adversarial patch with the similarity greater than the preset distance is output as the target adversarial patch.

[0117] The target adversarial patch can be a 3D adversarial patch, which can be made into an entity such as an eye patch, a face mask, etc., and can be referred to as a 3D face mask, a 3D prosthesis, a 3D eye patch, etc. The embodiments of the present application do not distinguish between them.

[0118] For ease of understanding, the above face image processing method is exemplified below in a specific application scenario. Specifically, the face feature coefficient is updated in each iteration, which is fed back to the 3DMM to generate a set of S and T, generate a 3D mask in one round, and at the same time render a 2D image for face verification task, so as to determine the attack effect of the 3D mask in this round, if it meets the standard, output the 3D mask; if it does not meet the standard, enter the next round of face feature coefficient update. It mainly includes the following three stages:

[0119] First stage: 3D face attack scene

[0120] In order to make the shape and texture of the generated 3D mask in the physical world achieve the effect of attacking the face recognition model, the following attack target function is designed in the embodiments of the present application:

[0121]

[0122] Where M represents a binary mask, which is used to represent the position of the candidate adversarial patch. And tri M represents the three-dimensional topological connection corresponding to the binary mask. represents the loss function, which is defined as in the evasion attack, and is defined as R represents the rendering process. In the actual algorithm, given tri M , then M is directly solved by projection transformation.

[0123] After solving the attack target function, a 3D adversarial mask can be generated using the new a, b, and d obtained in each iteration. The solving method can use a typical multi-step gradient iteration method, that is, in each step, the gradient of the above attack target function with respect to the solving target a * , b * , d * is solved, and then a * , b * , d * is updated. Details are not described herein.

[0124] Second stage: 3D face reconstruction

[0125] Given a face picture is a given face database. The 3D face modeling method is used to reconstruct the three-dimensional face information corresponding to x, and then a 3D mask can be obtained through 3D printing technology. Specifically, first, the shape S and texture T of the face x are reconstructed based on the above formula (1) and formula (2), that is, the face reconstruction image is obtained. Then, the 3D adversarial patch obtained in the first stage is superimposed on the face reconstruction image to obtain a 3D face adversarial sample.

[0126] To facilitate the face recognition task, the 3D face adversarial sample is first rendered into a 2D target adversarial sample using the above formula (3), that is, x * .

[0127] Third stage: face recognition task

[0128] For example, face recognition usually includes two sub-tasks, namely face verification and face recognition. The embodiments of the present application are mainly designed for the face verification task. For a face verification task, given a pair of face images {x First, the distance of the feature representation of the pair of images (i.e. D f (x a ,x b )) is calculated, and then it is judged whether the pair of images belong to the same person by whether the distance exceeds a threshold, that is, C(x a ,x b ) = I(D f (x a ,x b ) < δ), where I represents the indicator function and δ represents the threshold. When C(x a ,x b ) = 1, the pair of face images are recognized as the same person, otherwise they are recognized as different people. The embodiments of the present application mainly consider impersonation attacks, which aim to make the face recognition model recognize the target adversarial sample obtained in the second stage as a specific identity. Specifically, given a pair of face images {x a ,x b} belonging to different identities, the impersonation attack generates a target adversarial sample x a for x * , so that the face recognition model recognizes x * and x b as the same identity, that is, C(x * ,x b ) = 1, indicating that the attack is successful.

[0129] Compared with the prior art, in the embodiments of the present application, the candidate face shape and the candidate face texture are obtained according to the candidate face feature coefficient, the preset eigenface, the average face shape and the average face texture vector, and the candidate face feature coefficient is the personalized feature coefficient of the face of the first user as the attacked person based on the preset eigenface (a given fixed value), so the candidate face feature only represents the facial features of the first user, and then, on the one hand, the candidate face shape and the candidate face texture obtained based on the candidate face feature coefficient with such features can represent the key features of the original face and are close to the real face. Therefore, the candidate adversarial patch generated based on the candidate face shape and the candidate face texture has very good adversarial property, and compared with the 2D adversarial patch, the attack effect is not reduced due to the deformation or the face fitting degree; on the other hand, when the face verification task is performed based on the candidate adversarial patch, the attack effect of the candidate adversarial patch on the face recognition system can be effectively verified, and the target adversarial patch is determined through limited cycles in the embodiments of the present application, so that the target adversarial patch finally obtained fully considers the 3D transformation in the real scene, and therefore can better fit the real face. Compared with the 2D adversarial patch in the prior art, the problem of deformation of the physical adversarial patch is solved, so that the attack effect is ensured, and the success rate of the live detection module in the attack on the face recognition system is improved.

[0130] Optionally, in some embodiments of the present application, the image processing device can be in communication connection with at least one 3D printer, which is not limited in the embodiments of the present application. After the image processing device generates the 3D adversarial patch, the image processing device can send a printing instruction to the 3D printer to manufacture the 3D adversarial patch. Considering the printing loss, the printing parameters of the 3D printer can also be fine-tuned. Specifically, before the image processing device sends the printing instruction to the 3D printer, the printing parameters (key factors affecting the printing effect) of the 3D printer are obtained, and then the difference between the quality of the 3D adversarial patch printed based on the printing parameters and the theoretical parameters of the 3D adversarial patch is deduced. If each difference is greater than the corresponding threshold value, the printing parameters are adjusted or other 3D printers meeting the printing quality are switched.

[0131] The theoretical parameters include a, β and δ. The quality of the 3D adversarial patch printed based on the printing parameters can be inferred from the historical printing data. The threshold value can also be set or selected according to the experience value or the historical printing data, which is not limited.

[0132] This method allows for the printing of 3D adversarial patches that conform to theoretical parameters without upgrading or replacing the printer or overly relying on the printer's hardware, thus ensuring the effectiveness of the attack. Furthermore, it is compatible with various printers and allows for dynamic parameter adjustments to guarantee print quality and the effectiveness of the attack after applying the printed 3D adversarial patch.

[0133] In some implementations, the image processing described above can communicate with at least one 3D printer to print corresponding 3D adversarial patches for different business scenarios or needs.

[0134] Specifically, you can first determine the business scenario or business requirements, then determine the target printer that is compatible with the target model based on the business scenario or business requirements, and then send the printing command.

[0135] As can be seen, this solution can provide flexible and varied approaches for different objects or needs, and the printed 3D adversarial patches are more in line with actual needs, have lower costs, and are not limited to stringent implementation conditions.

[0136] Figures 1 to 4b Any technical feature mentioned in the embodiments corresponding to any one of the above also applies to the embodiments of this application. Figures 5 to 8 The corresponding implementation examples will not be repeated hereafter.

[0137] The foregoing has described a face image processing method according to an embodiment of this application. The following describes an image processing apparatus that performs the aforementioned face image processing method. This image processing apparatus can be a face recognition device.

[0138] See Figure 5 , Figure 5 This is a schematic diagram of a face recognition device according to an embodiment of this application, which can be applied to face recognition scenarios. The image processing device 50 in this embodiment of the application can achieve the corresponding to the above-described... Figures 1 to 4b The steps in the face image processing method executed by the image processing device 50 in any corresponding embodiment. The functions implemented by the image processing device 50 can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The image processing device 50 may include an input / output module 501 and a processing module 502. The functional implementation of the input / output module 501 and the processing module 502 can be found in [reference needed]. Figures 1 to 4b The operations performed in any of the corresponding embodiments will not be described in detail here.

[0139] In some implementations, the input / output module 501 can be used to obtain the candidate face feature coefficients of the first user;

[0140] The processing module 502 can be configured to obtain a candidate face shape and a candidate face texture according to the candidate face feature coefficient, the preset eigenface, the average face shape, and the average face texture vector obtained by the input and output module;

[0141] The processing module 502 is further configured to generate a candidate adversarial patch based on the candidate face shape and the candidate face texture, obtain a candidate adversarial sample, the candidate adversarial sample being obtained by superimposing the candidate adversarial patch on an attacker face image of a second user, and obtain a similarity between the candidate adversarial sample and a target face image of the first user.

[0142] If the similarity is less than a preset value, the candidate feature coefficient and the candidate adversarial patch are updated until the similarity between the candidate adversarial sample and the target face image is greater than the preset value, and the candidate adversarial patch when the similarity is greater than the preset distance is output as a target adversarial patch by the input and output module 501.

[0143] In some embodiments, the processing module 502 is specifically configured to:

[0144] determine a preset topological connection of a face three-dimensional patch;

[0145] perform projection transformation on the preset topological connection to obtain a binary mask, the binary mask being used to indicate a position to which the candidate adversarial patch is to be added in an attacker face image;

[0146] add the candidate adversarial patch to the attacker face image according to the position indicated by the binary mask to obtain the candidate adversarial sample.

[0147] In some embodiments, the processing module 502 is specifically configured to:

[0148] obtain a target face image;

[0149] input the target face image into a three-dimensional morphable model, and calculate the candidate face feature coefficient according to a preset average face shape, an average face texture vector, a preset eigenface, and the target face image in the three-dimensional morphable model;

[0150] obtain the candidate face shape according to the average face shape, the preset eigenface, and the candidate face feature coefficient, and obtain the candidate face texture according to the preset eigenface and the candidate face feature coefficient.

[0151] In some embodiments, the attacker face image is a first face image of the second user from the target image set; and the processing module 502 is specifically configured to:

[0152] obtain the candidate adversarial patch and the attacker face image;

[0153] superimpose the candidate adversarial patch and the first face image to obtain a 3D face adversarial sample;

[0154] map the 3D face adversarial sample to the candidate adversarial sample in 2D.

[0155] In some embodiments, the processing module 502 is specifically configured to:

[0156] obtain a preset topological connection of a light parameter value, a shooting angle, and a face three-dimensional patch;

[0157] input the 3D face adversarial sample into a renderer according to the light parameter value, the shooting angle, and the preset topological connection to obtain the candidate adversarial sample.

[0158] In some embodiments, the processing module 502 is specifically configured to:

[0159] obtain a second face image collected by a shooting device in real time through the input and output module 501, the second face image being an image collected after a second user wears a real 3D adversarial patch;

[0160] use the second face image as the candidate adversarial sample.

[0161] In some embodiments, the processing module 502 can also be configured to perform at least one of the following:

[0162] determine a business requirement, and determine a target printer that adapts to a target model according to the business requirement;

[0163] Alternatively, obtain historical printing data, derive a difference between a quality of a 3D adversarial patch printed based on the historical printing data and a theoretical parameter of the 3D adversarial patch according to the historical printing data, and if each difference is greater than a corresponding threshold value, adjust a printing parameter of a 3D printer or switch the 3D printer.

[0164] In this embodiment, the candidate face shape and candidate facial texture are obtained based on the candidate face feature coefficient, preset feature face, average face shape and average face texture vector. The candidate face feature coefficient is a personalized feature coefficient obtained by the first user's face as the attacked based on the preset feature face (given fixed value). Therefore, the candidate face feature only represents the facial features of the first user. Thus, on one hand, the candidate face shape and candidate facial texture obtained based on the candidate face feature coefficient with such characteristics can represent the key features of the original face and fit the real face. Therefore, the candidate adversarial patch generated based on the candidate face shape and texture has very good adversarial power. Compared with 2D adversarial patches, it will not reduce the attack effect due to deformation or facial fit. On the other hand, when a face verification task is performed based on the candidate adversarial patch, the attack effect of the candidate adversarial patch on the face recognition system can be effectively verified. Furthermore, the target adversarial patch is determined by a finite loop in this embodiment. Therefore, the final target adversarial patch fully considers the 3D transformation in the real scene, thus better fitting the real face. Compared with the 2D adversarial patch in the prior art, it not only solves the problem of deformation of the generated physical adversarial patch, thereby ensuring its attack effect, but also improves the success rate of attacking the liveness detection module in the face recognition system.

[0165] The image processing device 50 in the embodiments of this application has been described above from the perspective of modular functional entities. The image processing device 50 in the embodiments of this application will be described below from the perspective of hardware processing.

[0166] It should be noted that, in the embodiments of this application Figure 5 In the embodiments shown, the physical device corresponding to the input / output module 501 can be an input / output unit, transceiver, radio frequency circuit, communication module, and output interface, etc., and the physical device corresponding to the processing module 502 can be a processor. Figure 5 The image processing device 50 shown may have, for example: Figure 6 The structure shown, when Figure 5 The image processing apparatus 50 shown has, for example: Figure 6 When the structure shown is used, Figure 6 The processor and transceiver in the image processing device 50 can perform the same or similar functions as the input / output module 501 and processing module 502 provided in the aforementioned device embodiment corresponding to the image processing device 50. Figure 6 The memory stores the computer programs that the processor needs to call when executing the above face image processing method.

[0167] Figure 5 The image processing device 50 shown may have, for example: Figure 6 The structure shown, when Figure 5The image processing apparatus 50 shown has a structure as shown in Figure 6 When the structure shown in Figure 6 The processor in the structure shown in the foregoing device embodiment of the device corresponding to the processing module 502 can realize the same or similar functions as the foregoing device embodiment of the device corresponding to the processing module 502, Figure 6 The transceiver in the structure shown in the foregoing device embodiment of the device corresponding to the input and output module 501 can realize the same or similar functions as the foregoing device embodiment of the device corresponding to the input and output module 501, Figure 6 The memory in the structure shown in the foregoing device embodiment of the device corresponding to the input and output module 501 can realize the same or similar functions as the foregoing device embodiment of the device corresponding to the input and output module 501, Figure 5 The transceiver module 501 and the processing module 502 in the embodiment shown correspond to the entity device. The processing module 502 corresponds to the entity device, which can be an input and output interface.

[0168] The embodiment of the present application also provides another terminal device, as shown in Figure 7 For ease of illustration, only parts related to the embodiment of the present application are shown, and specific technical details are not disclosed. Please refer to the method part of the embodiment of the present application. The terminal device can be any terminal device, including a mobile phone, a tablet computer, a personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA), a sales terminal (English full name: Point of Sales, English abbreviation: POS), a vehicle-mounted computer, etc. Taking the terminal as a mobile phone as an example:

[0169] Figure 7 The diagram shown is a block diagram of part of the structure of the mobile phone related to the terminal device provided by the embodiment of the present application. Referring to Figure 7 , the mobile phone includes: radio frequency (English full name: Radio Frequency, English abbreviation: RF) circuit 610, memory 620, input unit 630, display unit 640, sensor 650, audio circuit 660, wireless fidelity (English full name: wireless fidelity, English abbreviation: WiFi) module 670, processor 680, and power supply 690, etc. Those skilled in the art can understand Figure 7 that the structure of the mobile phone shown in the foregoing embodiment of the present application does not constitute a limitation on the mobile phone, and can include more or fewer components than those shown, or combine certain components, or different component arrangements.

[0170] The various constituent components of the mobile phone will be specifically introduced below: Figure 7

[0171] ​The RF circuit 610 can be used for receiving and sending signals in the process of information or communication, especially, receiving the downlink information from the base station and processing it by the processor 680; in addition, sending the uplink data to the base station. Generally, the RF circuit 610 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 610 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0172] The memory 620 can be used to store software programs and modules, and the processor 680 can execute various function applications and data processing of the mobile phone by running the software programs and modules stored in the memory 620. The memory 620 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 620 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0173] The input unit 630 can be used to receive inputted digital or character information, and to generate key signal input with respect to user setting of the mobile phone and control of function. Specifically, the input unit 630 can include a touch panel 631 and other input device 632. The touch panel 631, also called a touch screen, can collect a touch operation (such as an operation of a user using a finger, a stylus, or any suitable object or accessory on or near the touch panel 631) of the user on or near it, and drive a corresponding connection device according to a pre-set program. Optionally, the touch panel 631 can include two parts of a touch detection device and a touch controller. Among them, the touch detection device detects the touch position of the user and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, and converts it into touch coordinates and sends it to the processor 680, and can also receive the command from the processor 680 and execute it. In addition, the touch panel 631 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 631, the input unit 630 can also include other input device 632. Specifically, the other input device 632 can include one or more of a physical keyboard, a function key (such as a volume control button, an on-off button, etc.), a trackball, a mouse, a joystick, etc.

[0174] The display unit 640 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 640 can include a display panel 641, which can be configured in the form of a liquid crystal display (English full name: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (English full name: Organic Light-Emitting Diode, English abbreviation: OLED), etc. Further, the touch panel 631 can cover the display panel 641, and when the touch panel 631 detects a touch operation on or near it, it is transmitted to the processor 680 to determine the type of touch event, and then the processor 680 provides corresponding visual output on the display panel 641 according to the type of touch event. Although in the Figure 6 In some embodiments, the touch panel 631 and the display panel 641 can be integrated to realize the input and output functions of the mobile phone.

[0175] The phone can also include at least one sensor 650, such as an optical sensor, a motion sensor, and other sensors. Specifically, the optical sensor can include an ambient light sensor to adjust the brightness of the display panel 641 according to the brightness of ambient light, and a proximity sensor to turn off the display panel 641 and / or the backlight when the phone is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude and direction of the acceleration in each direction (generally three axes), and when at rest, the magnitude and direction of gravity, which can be used for applications that identify the phone posture (such as switching between landscape and portrait screens, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), and the like. As for other sensors that the phone can also be configured, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, will not be described here.

[0176] The audio circuit 660, the speaker 661, and the microphone 662 can provide an audio interface between the user and the phone. The audio circuit 660 can convert the received audio data into an electrical signal, transmit it to the speaker 661, and convert it into a sound signal output by the speaker 661; on the other hand, the microphone 662 converts the collected sound signal into an electrical signal, which is received by the audio circuit 660 and converted into audio data, which is then processed by the processor 680 and transmitted to another phone via the RF circuit 610, or output to the memory 620 for further processing.

[0177] Wi-Fi belongs to a short-range wireless transmission technology, and the phone can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 670, which provides users with wireless broadband Internet access. Although Figure 6 The Wi-Fi module 670 is shown, but it is understood that it does not belong to the necessary components of the phone, and can be omitted as needed without changing the essence of the application.

[0178] The processor 680 is the control center of the phone, which connects all parts of the phone through various interfaces and lines, executes various functions of the phone and processes data by running or executing software programs and / or modules stored in the memory 620, and calling data stored in the memory 620, thereby monitoring the phone as a whole. Optionally, the processor 680 can include one or more processing units; preferably, the processor 680 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 680.

[0179] The mobile phone further includes a power supply 690 (such as a battery) for supplying power to each component, which can be logically connected to the processor 680 through a power management system, so as to realize functions such as charge management, discharge management, and power consumption management through the power management system.

[0180] Although not shown, the mobile phone can further include a camera, a Bluetooth module, and the like, which will not be described herein.

[0181] In the embodiments of the present application, the processor 680 included in the mobile phone further has a function of controlling the face image processing method flow performed by the image processing apparatus 50 shown in Figure 5 .

[0182] Figure 8 is a server structure schematic diagram provided by the embodiments of the present application. The server 720 can have a large difference due to different configurations or performances, and can include one or more central processing units (English full name: central processing units, English abbreviation: CPU) 722 (for example, one or more processors) and a memory 732, one or more storage media 730 (for example, one or more mass storage devices) for storing application programs 742 or data 744. Among them, the memory 732 and the storage medium 730 can be temporary storage or persistent storage. The programs stored in the storage medium 730 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the server. Further, the central processing unit 722 can be configured to communicate with the storage medium 730 and execute a series of instruction operations in the storage medium 730 on the server 720.

[0183] The server 720 can further include one or more power supplies 726, one or more wired or wireless network interfaces 750, one or more input / output interfaces 758, and / or one or more operating systems 741, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and the like.

[0184] The steps performed by the server in the above embodiments can be based on the structure of the server 720 shown in Figure 8 . For example, the steps performed by the first node shown in Figure 5 in the above embodiments can be based on the server structure shown in Figure 8 . For example, the processor 7202 performs the following operations by invoking instructions in the memory 732:

[0185] Obtain the candidate face feature coefficient of the first user through the input / output interface 758;

[0186] The candidate face feature coefficient, the preset eigenface, the average face shape and the average face texture vector obtained according to the input and output interface 758 are used to obtain a candidate face shape and a candidate face texture.

[0187] A candidate adversarial patch is generated based on the candidate face shape and the candidate face texture. A candidate adversarial sample is obtained, which is obtained by superimposing the candidate adversarial patch on an attacker face image of a second user. The similarity between the candidate adversarial sample and the target face image of the first user is obtained.

[0188] If the similarity is less than a preset value, the candidate feature coefficient and the candidate adversarial patch are updated until the similarity between the candidate adversarial sample and the target face image is greater than the preset value. The candidate adversarial patch when the similarity is greater than the preset distance is output as a target adversarial patch through the input and output interface 758.

[0189] In the above embodiments, the description of each embodiment focuses on different aspects. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0190] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and module described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0191] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. Actual implementation can have another division manner. For example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0192] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, i.e. they can be located in one place or distributed on multiple network modules. According to actual needs, some or all of the modules can be selected to achieve the purpose of the embodiments.

[0193] In addition, each of the functional modules in each of the embodiments of the present application can be integrated in one processing module, or each of the modules can exist physically independently, or two or more of the modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, the integrated module can be stored in a computer readable storage medium.

[0194] In the above embodiments, all or part of the embodiments can be realized by software, hardware, firmware, or any combination thereof. When realized by software, all or part of the embodiments can be realized in the form of a computer program product.

[0195] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be stored by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0196] The above describes the technical solutions provided by the embodiments of the present application in detail. The principles and implementation manners of the embodiments of the present application are described by using specific examples. The above descriptions are only used to help understand the method and core idea of the embodiments of the present application; meanwhile, for those skilled in the art, according to the ideas of the embodiments of the present application, the specific implementation manners and application ranges will be changed. Therefore, the content of the present application should not be understood as a limitation of the embodiments of the present application.

Claims

1. A face image processing method, characterized by, The method comprises: obtaining candidate face feature coefficients of a first user; obtaining candidate face shape and candidate face texture according to the candidate face feature coefficients, preset feature face, average face shape and average face texture vector; generating a candidate adversarial patch based on the candidate face shape and the candidate face texture; obtaining a candidate adversarial sample, which is obtained by superimposing the candidate adversarial patch and an attacker face image of a second user; obtaining the similarity between the candidate adversarial sample and a target face image of the first user; if the similarity is less than a preset value, updating the candidate face feature coefficients and the candidate adversarial patch until the similarity between the candidate adversarial sample and the target face image is greater than the preset value, and outputting the candidate adversarial patch when the similarity is greater than the preset value as a target adversarial patch.

2. The method of claim 1, wherein, The method comprises: determining a preset topological connection of a face three-dimensional surface patch; projective transformation is performed on the preset topological connection to obtain a binary mask, and the binary mask is used to indicate a position where the candidate adversarial patch is to be added to the attacker face image; adding the candidate adversarial patch to the attacker face image according to the position indicated by the binary mask to obtain the candidate adversarial sample.

3. The method according to claim 1 or 2, characterized in that, The method comprises: obtaining a target face image; inputting the target face image into a three-dimensional deformation model, and calculating the candidate face feature coefficients according to the average face shape, average face texture vector, preset feature face and the target face image in the three-dimensional deformation model; The method comprises: obtaining the candidate face shape according to the average face shape, the preset feature face and the candidate face feature coefficients, and obtaining the candidate face texture according to the preset feature face and the candidate face feature coefficients.

4. The method of claim 3, wherein, The attacker face image is a first face image of the second user from the target face image set; the method comprises: obtaining the candidate adversarial patch and the attacker face image; superimposing the candidate adversarial patch and the first face image to obtain a 3D face adversarial sample; mapping the 3D face adversarial sample into a 2D candidate adversarial sample.

5. The method of claim 4, wherein, The method comprises: obtaining a light parameter value, a shooting angle and a preset topological connection of a face three-dimensional surface patch; inputting the 3D face adversarial sample into a renderer according to the light parameter value, the shooting angle and the preset topological connection to obtain the candidate adversarial sample.

6. The method of claim 3, wherein, The method comprises: obtaining a second face image collected by a shooting device in real time, the second face image being an image collected after a second user wears a physical 3D adversarial patch; taking the second face image as the candidate adversarial sample.

7. The method according to any one of claims 1-2, 4-6, characterized in that, Before outputting the target adversarial patch, the method further comprises at least one of the following: Determine a service requirement, determine a target printer that fits a target model according to the service requirement; Or, obtain historical printing data, derive a difference between a quality of a 3D adversarial patch printed based on the historical printing data and a theoretical parameter of the 3D adversarial patch according to the historical printing data, if each difference is greater than a respective threshold value, adjust a printing parameter of the 3D printer or switch the 3D printer correspondingly.

8. An image processing apparatus characterized by comprising: The image processing device comprises: An input / output module configured to obtain candidate face feature coefficients of a first user; A processing module configured to obtain candidate face shapes and candidate face texture based on the candidate face feature coefficients, a pre-set feature face, an average face shape and an average face texture vector obtained by the input / output module; The processing module is further configured to generate a candidate adversarial patch based on the candidate face shape and the candidate face texture, obtain a candidate adversarial sample by superimposing the candidate adversarial patch on an attacker face image of a second user, and obtain a similarity between the candidate adversarial sample and a target face image of the first user; If the similarity is less than a pre-set value, update the candidate face feature coefficients and the candidate adversarial patch until the similarity between the candidate adversarial sample and the target face image is greater than the pre-set value, and output the candidate adversarial patch when the similarity is greater than the pre-set value as a target adversarial patch through the input / output module.

9. A face recognition apparatus, characterized by comprising: The device comprises: At least one processor, memory and transceiver; The memory is configured to store a computer program, and the processor is configured to call the computer program stored in the memory to execute the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It comprises instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1-7.

Images