A face image anonymization method, system, device and medium

Abstract

The application discloses a face image anonymization method, system, device and medium, and relates to the field of image processing.The technical scheme points are as follows:an original face image is acquired, and an encoding feature vector of the original face image is extracted;the encoding feature vector is input into a diffusion model generated by multiple time steps and repeated iterations to generate an anonymous face image;the diffusion model comprises a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image;in each time step, an anonymous loss function is also introduced, the cosine similarity between the face image generated in the current time step and the original face image is calculated according to the anonymous loss function, the gradient of the cosine similarity to the face image in the current time step is calculated, the face image generated in the current time step is determined according to the gradient of the face image in the current time step, and the face image is taken as the input of the next time step; when the anonymous loss calculation of all time steps is completed, the anonymous face image is generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, more particularly, it relates to a face image anonymization method, system, device and medium. BACKGROUND

[0002] Face anonymization refers to a process of processing face information to make it impossible to identify the original identity, and the core idea is to eliminate or hide the identity sensitive information in the image by using de-identification and other technical means.

[0003] According to different anonymization goals, face anonymization technology can be divided into methods based on visual content modification, methods based on visual content preservation, and visual recoverable anonymization methods. Among them, visual content modification means that the visual effect is different before and after image anonymization in the view of human eyes; visual content preservation means that the visual appearance of the face does not change before and after anonymization, but the face recognition system considers that the identity of the image before and after anonymization is inconsistent. Visual recoverable anonymization method means that the anonymized image can be restored to the image before anonymization, and the whole process is "reversible". Traditional face anonymization methods include blurring, mosaicking and occlusion, that is, anonymization is achieved by covering the face area. With the development of deep generative network technology, face anonymization can generate a false face while keeping the identity-independent attributes such as facial expression and posture unchanged, without affecting the visual quality of the face after anonymization.

[0004] In the image anonymization method, the related technology can remove the face identity information after anonymization by designing face classification adversarial loss, reconstruction loss, background consistency loss and photo-realistic loss, and when a correct password is given, the original face can be restored, and when an incorrect password is given, a false identity and natural face is returned, without sacrificing face privacy, realizing reversible operation of anonymization. However, this method cannot control the anonymization strength of the anonymized face image, which will result in a high similarity between the generated anonymized face image and the original face image, and cannot effectively hide the identity sensitive information in the image. SUMMARY

[0005] The purpose of the present application is to provide a face image anonymization method, system, device and medium, which can control the anonymization strength of the anonymized face image.

[0006] In a first aspect of the present application, a face image anonymization method is provided, the method comprising:

[0007] obtaining an original face image and extracting an encoding feature vector of the original face image;

[0008] input the code representation vector into a diffusion model generated by iterative iteration of multiple time steps to generate an anonymous face image; wherein the diffusion model comprises a reconstruction loss function, and the reconstruction loss parameter is a reconstruction loss between the original face image and the face image; in each time step, an anonymity loss function is also introduced, the cosine similarity between the face image generated at the current time step and the original face image is calculated according to the anonymity loss function, the gradient of the cosine similarity to the face image at the current time step is calculated, and the face image generated at the current time step is determined according to the gradient of the face image at the current time step, and the face image is taken as the input of the next time step, and when the anonymity loss calculation of all time steps is completed, the anonymous face image is generated.

[0009] In an implementation scheme, the code representation vector of the original face image is extracted by a downsampling network encoder.

[0010] In an implementation scheme, the expression of the reconstruction loss function is: wherein ∈ represents the real noise added noise of the noise prediction network, ∈ θ represents the noise predicted by the noise prediction model according to the input information, ∈ θ (·) represents the input of the noise prediction model, t represents the noise added step, x t represents the original face image x org According to the noise image obtained by the real noise added noise ∈ and the noise added step t, z org represents the original face image x org The code representation vector extracted by the encoder.

[0011] In an implementation scheme, the expression of the anonymity loss function is: wherein t represents the current time step, F θ (·) represents a pre-trained face recognition model, x org represents the original face image, represents the face image generated at the current time step, F θ (x org ) represents the identity feature vector of the original face image, represents the identity feature vector of the face image generated at the current time step.

[0012] In an implementation scheme, the face image generated at the current time step is determined according to the gradient of the face reconstruction image at the current time step and the gradient, specifically: a gradient guide intensity factor of the identity feature difference between the generated anonymous face image and the original face image is preset;

[0013] The face image generated at the current time step is determined by the difference between the face reconstruction image at the current time step and the product of the gradient guide intensity factor and the gradient.

[0014] In an implementation, the method further comprises:

[0015] The original face image is reversely coded by a diffusion model to extract a random representation vector;

[0016] The coding representation vector of the anonymous face image is extracted by a down-sampling network encoder, and a residual representation vector is obtained by calculating the difference between the original coding representation vector;

[0017] The random representation vector and the residual representation vector are input into the diffusion model to generate a recovered face image.

[0018] In a second aspect, the present application provides a face image anonymization system, which comprises:

[0019] A vector extraction module is configured to obtain an original face image and extract a coding representation vector of the original face image;

[0020] An image anonymization module is configured to input the coding representation vector into a diffusion model generated by iterative iteration of multiple time steps to generate an anonymous face image; wherein the diffusion model comprises a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; in each time step, an anonymity loss function is further introduced, the cosine similarity between the face image generated in the current time step and the original face image is calculated according to the anonymity loss function, the gradient of the face image in the current time step is calculated according to the cosine similarity, the face image generated in the current time step is determined according to the gradient of the face image in the current time step, and the face image is taken as the input of the next time step, and the anonymous face image is generated after the anonymity loss calculation of all time steps is completed.

[0021] In an implementation, the coding representation vector of the original face image is extracted by a down-sampling network of an encoder.

[0022] In a third aspect, the present application provides an electronic device, which comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the face image anonymization method provided in the first aspect of the present application are implemented.

[0023] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein when the computer program is executed by a processor, the steps of the face image anonymization method provided in the first aspect of the present application are implemented.

[0024] Compared with the prior art, the present application has the following beneficial effects:

[0025] The face image anonymization method provided by the present application comprises the following steps: extracting an encoding feature vector of an original face image, reconstructing a face reconstruction image according to the encoding feature vector, calculating the cosine similarity between the face reconstruction image and the original face image at a current time step through an anonymity loss function in a diffusion model generated by repeatedly iterating a plurality of time steps, calculating the gradient of the cosine similarity with respect to the face reconstruction image at the current time step, determining the face image generated at the current time step according to the face reconstruction image at the current time step and the gradient, and controlling the anonymity strength of the anonymous face image by adjusting the gradient guide strength factor in the anonymity loss function, so that the similarity between the anonymous face image and the original face image is low, thereby effectively hiding the identity sensitive information in the image. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are included to provide a further understanding of the embodiments of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and together with the description serve to explain the principles of the present application. In the drawings:

[0027] Figure 1 A flowchart of a face image anonymization method provided by an embodiment of the present application is shown in the figure.

[0028] Figure 2 A principle block diagram of a face image anonymization system provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0029] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be given to the present application in combination with embodiments and drawings, and the illustrative embodiments of the present application and the description thereof are only used to explain the present application and should not be regarded as limiting the present application.

[0030] It should be noted that the term "include" or "may include" used in various embodiments of the present application indicates the existence of the claimed function, operation or element, and does not limit the addition of one or more functions, operations or elements. In addition, as used in various embodiments of the present application, the terms "include", "have" and their synonyms only mean to indicate the presence of a specific feature, number, step, operation, element, component or combination of the foregoing, and should not be understood as first excluding the presence or addition of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing.

[0031] In various embodiments of the present application, the expression "or" or "at least one of B or / and C" includes any combination or all combinations of the listed terms. For example, the expression "B or C" or "at least one of B or / and C" can include B, can include C, or can include both B and C.

[0032] It should be understood that in the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly and specifically limited.

[0033] Reference is made to Figure 1 , Figure 1 A flowchart of a face image anonymization method provided by an embodiment of the present application is shown in FIG. 1, and the method comprises the following steps.

[0034] S101, an original face image is obtained, and an encoding feature vector of the original face image is extracted.

[0035] In this embodiment, the original face image is an unprocessed face image collected, for example, the original face image can be a face image collected by a community access control or a monitoring camera, or a face image collected by a face payment software. The original face image can be collected in real time or non-real time, and the present embodiment does not make special limitations on this.

[0036] Secondly, the original face image can also be cropped and aligned into a standard face image. Specifically, an encoding feature vector of the original face image is extracted by a down-sampling network encoder. That is, the encoder adopts a down-sampling network of U-Net, which is used to extract high-level semantic information of the face image, and the high-level semantic information is the encoding feature vector.

[0037] S102, the encoding feature vector is input into a diffusion model generated by multiple time steps iteratively, to generate an anonymous face image; wherein the diffusion model comprises a reconstruction loss function, and the reconstruction loss parameter is a reconstruction loss between the original face image and the face image; in each time step, an anonymity loss function is also introduced, the cosine similarity between the face image generated in the current time step and the original face image is calculated according to the anonymity loss function, the gradient of the current time step face image is calculated according to the cosine similarity, the face image generated in the current time step is determined according to the gradient of the current time step face image, and the face image is taken as the input of the next time step, and when the anonymity loss calculation of all time steps is completed, the anonymous face image is generated.

[0038] In this embodiment, in order to enable the diffusion model to reconstruct the face image according to the input encoding feature vector, the present embodiment also introduces a reconstruction loss function in the diffusion model, so that the diffusion model has the ability to reconstruct the original face image, that is, by updating the parameters of the encoder and the diffusion model, the loss function adopted is the reconstruction loss between the original face image and the generated face image, and the expression of the reconstruction loss function is: wherein ∈ represents the true noise added by the noise prediction network, and ∈ θrepresents the noise predicted by the noise prediction model according to the input information, ∈ θ (·) represents the input of the noise prediction model, t represents the noise adding step, x t represents the original face image x org represents the noise image obtained according to the real noise ∈ and the noise adding step t, z org represents the original face image x org The encoding representation vector extracted by the encoder.

[0039] In further embodiments, the generation process of the diffusion model is a process repeatedly iterated by T time steps, so that an anonymity loss function can be introduced at each time step, the cosine similarity between the identity feature vector of the face image generated at the current time step t and the identity feature vector of the original face image is calculated first, and then the gradient of the face image generated at the time step t with respect to the similarity is calculated, and the gradient guides the image generated at the current time step t to be far away from the original face image. Therefore, the pre-trained encoder, the pre-trained generation model, and the anonymity loss function introduced in the generation stage of the generation model jointly act, and the generated face image can reconstruct the background and face posture of the original image, while changing the facial features, achieving the effect of face anonymization.

[0040] Specifically, the expression of the anonymity loss function is: Wherein, t represents the current time step, F θ (·) represents the pre-trained face recognition model, x org represents the original face image, represents the face image generated at the current time step, F θ (x org ) represents the identity feature vector of the original face image, represents the identity feature vector of the face image generated at the current time step.

[0041] In some embodiments, the face image generated at the current time step is determined according to the gradient of the face reconstruction image at the current time step and, specifically: a gradient guidance intensity factor representing the difference between the generated anonymous face image and the original face image is determined; the face image generated at the current time step is determined by the difference between the face reconstruction image at the current time step and the product of the gradient guidance intensity factor and the gradient.

[0042] Specifically, the cosine similarity between the identity feature vector of the generated face image and the identity feature vector of the original face image is calculated by the loss function The gradient grad of the face image generated at the current time step with respect to the similarity is calculated, and the expression is: t which guides the face image generated at the current time step t to be far away from the original face image.

[0043] Here, γ represents the gradient guidance strength factor. The value of γ determines the difference in identity features between the generated anonymous face image and the original face image. When γ is set very small, the guidance strength is weak, and the generated anonymous face image is similar to the original face image. When γ is set very large, the guidance strength is effective, and the generated anonymous face image shows differences from the original face image. When the similarity is very low, the anonymization effect is achieved. Therefore, by adjusting the gradient guidance strength factor, the anonymization strength of the anonymous face image can be controlled, resulting in a low similarity between the anonymous face image and the original face image, thereby effectively hiding sensitive identity information in the image.

[0044] In some embodiments, the method further includes: performing reverse encoding processing on the original face image using a diffusion model to extract a random representation vector; extracting the encoded representation vector of the anonymous face image using a downsampling network encoder, and calculating the difference between the encoded representation vector and the original encoded representation vector to obtain a residual representation vector; and inputting the random representation vector and the residual representation vector into the diffusion model to generate a restored face image.

[0045] In this embodiment, the encoder and diffusion model possess reconstruction capabilities, also known as restoration capabilities. In this embodiment, the restoration process requires specific keys KeyZ and KeyR, where key KeyZ is derived from the original face image x. org The key KeyR is derived from the encoded representation vector extracted by the encoder and is composed of the original face image x. org Random representation vectors extracted through inverse encoding using a diffusion model.

[0046] Please refer to Figure 2 , Figure 2 A schematic diagram of a face image anonymization system provided in an embodiment of the present invention is shown below. Figure 2 As shown, the system includes:

[0047] Vector extraction module 210 is used to acquire the original face image and extract the encoded representation vector of the original face image;

[0048] The image anonymization module 220 is configured to input the encoding feature vector into a diffusion model generated by iterative iteration of multiple time steps to generate an anonymous face image; wherein the diffusion model comprises a reconstruction loss function, and the reconstruction loss parameter is a reconstruction loss between the original face image and the face image; in each time step, an anonymous loss function is further introduced, the cosine similarity between the face image generated in the current time step and the original face image is calculated according to the anonymous loss function, the gradient of the cosine similarity to the face image in the current time step is calculated, the face image generated in the current time step is determined according to the gradient of the face image in the current time step, and the face image is taken as the input of the next time step, and when the anonymous loss calculation of all time steps is completed, the anonymous face image is generated.

[0049] It can be seen that the embodiment of the present application provides a face image anonymization system, which extracts the encoding feature vector of the original face image, reconstructs the face reconstruction image according to the encoding feature vector, and then combines the diffusion model generated by iterative iteration of multiple time steps, calculates the cosine similarity between the face reconstruction image in the current time step and the original face image through the anonymous loss function, calculates the gradient of the cosine similarity to the face reconstruction image in the current time step, determines the face image generated in the current time step according to the gradient of the face reconstruction image in the current time step, adjusts the gradient guide intensity factor in the anonymous loss function to control the anonymous intensity of the anonymous face image, so that the similarity between the anonymous face image and the original face image is low, and the identity sensitive information in the image is effectively hidden.

[0050] In some embodiments, the encoding feature vector of the original face image is extracted through a down-sampling network of an encoder.

[0051] The embodiment of the present application also provides an electronic device, which comprises a processor, a memory, a communication interface and at least one communication bus for connecting the processor, the memory and the communication interface. The memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (PROM) or portable read-only memory (CD-ROM), which is used for related instructions and data.

[0052] The communication interface is configured to receive and send data. The processor can be one or more CPUs, and in the case of one CPU, the CPU can be a single-core CPU or a multi-core CPU. The processor in the electronic device is configured to read one or more programs stored in the memory and perform the following operations: obtaining an original face image and extracting an encoded feature vector of the original face image; inputting the encoded feature vector into a diffusion model generated by multiple time steps of iterative iteration to generate an anonymous face image; wherein the diffusion model includes a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; in each time step, an anonymity loss function is also introduced, the cosine similarity between the face image generated in the current time step and the original face image is calculated according to the anonymity loss function, the gradient of the cosine similarity to the face image in the current time step is calculated, the face image generated in the current time step is determined according to the gradient of the face image in the current time step, and the face image is taken as the input of the next time step, and when the anonymity loss calculation of all time steps is completed, the anonymous face image is generated.

[0053] It should be noted that the specific implementation of each operation can be described above Figure 1 The electronic device can be configured to perform a face image anonymization method according to the above method embodiments of the present application, and the specific details are not described here.

[0054] The computer readable storage medium in the embodiments of the present disclosure is a memory device in the computer device, and is used to store programs and data. It can be understood that the computer readable storage medium herein can include an internal storage medium in the computer device, and of course can include an extended storage medium supported by the computer device. The computer readable storage medium provides a storage space, and the storage space stores an operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory or a non-volatile memory such as at least one disk memory. The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the face image anonymization method in the above embodiments. Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0055] The above detailed description is further detailed to explain the purpose, technical solutions and beneficial effects of the present disclosure. It should be understood that the above detailed description is only a specific embodiment of the present disclosure and is not used to limit the protection scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A face image anonymization method, characterized by, The method comprises: obtaining an original face image and extracting an encoded feature vector of the original face image; inputting the encoded feature vector into a diffusion model generated by iterative iteration of multiple time steps to generate an anonymous face image; wherein the diffusion model comprises a reconstruction loss function, and the reconstruction loss function is a reconstruction loss between the original face image and the face image; in each time step, an anonymous loss function is also introduced, the cosine similarity between the face image generated in the current time step and the original face image is calculated according to the anonymous loss function, the gradient of the cosine similarity to the face image in the current time step is calculated, the face image generated in the current time step is determined according to the gradient of the face image in the current time step, and the face image is taken as the input of the next time step, and when the anonymous loss calculation of all time steps is completed, the anonymous face image is generated; wherein an expression of the anonymous loss function is: wherein, denotes a current time step, denotes a pre-trained face recognition model, denotes an original face image, denotes a face image generated at a current time step, denotes an identity feature vector of the original face image, denotes an identity feature vector of the face image generated at the current time step; determining the face image generated in the current time step according to the gradient of the face reconstruction image in the current time step, specifically: presetting a gradient guide intensity factor representing the identity feature difference between the generated anonymous face image and the original face image; determining the face image generated in the current time step through the difference between the face reconstruction image in the current time step and the product of the gradient guide intensity factor and the gradient.

2. The face image anonymization method of claim 1, wherein, The encoded feature vector of the original face image is extracted through a downsampling network encoder.

3. The method of claim 1, wherein, The expression of the reconstruction loss function is: wherein, represents the real noisy noise of the noise prediction network, represents the noise predicted by the noise prediction model according to the input information, represents the input of the noise prediction model, represents the noise adding step, represents the original face image According to the real noisy noise and the noise adding step The noise image obtained, represents the original face image The encoding feature vector extracted by the encoder.

4. The face image anonymization method of claim 1, wherein, The method further comprises: extracting a random feature vector by performing reverse encoding processing on the original face image through the diffusion model; extracting an encoded feature vector of the anonymous face image through a downsampling network encoder, and calculating the difference between the original encoded feature vector to obtain a residual feature vector; inputting the random feature vector and the residual feature vector into the diffusion model to generate a recovered face image.

5. A face image anonymization system, comprising: The system comprises: a vector extraction module configured to obtain an original face image and extract an encoded feature vector of the original face image; an image anonymization module configured to input the encoded feature vector into a diffusion model generated by iterative iteration of multiple time steps to generate an anonymous face image; wherein the diffusion model comprises a reconstruction loss function, and the reconstruction loss function is a reconstruction loss between the original face image and the face image; in each time step, an anonymous loss function is also introduced, the cosine similarity between the face image generated in the current time step and the original face image is calculated according to the anonymous loss function, the gradient of the cosine similarity to the face image in the current time step is calculated, the face image generated in the current time step is determined according to the gradient of the face image in the current time step, and the face image is taken as the input of the next time step, and when the anonymous loss calculation of all time steps is completed, the anonymous face image is generated; wherein, the expression of the anonymous loss function is: wherein, denotes the current time step, denotes a pre-trained face recognition model, denotes an original face image, denotes a face image generated at the current time step, denotes an identity feature vector of the original face image, denotes an identity feature vector of the face image generated at the current time step; determining the face image generated in the current time step according to the gradient of the face reconstruction image in the current time step, specifically: presetting a gradient guide intensity factor representing the identity feature difference between the generated anonymous face image and the original face image; determining the face image generated in the current time step through the difference between the face reconstruction image in the current time step and the product of the gradient guide intensity factor and the gradient.

6. The face image anonymization system of claim 5, wherein, The encoded feature vector of the original face image is extracted through a downsampling network encoder of the encoder.

7. An electronic device, comprising: The electronic device comprises a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the face image anonymization method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program, when executed by the processor, implements the steps of the face image anonymization method according to any one of claims 1 to 4.

Images