Face image anonymization method, system and device and medium

Abstract

The invention discloses a face image anonymization method, system and device and a medium, and relates to the field of image processing, and the key points of the technical scheme are that the method comprises the steps: obtaining an original face image, and extracting a coding representation vector of the original face image; inputting the coding representation vector into a diffusion model generated by repeated iteration of a plurality of time steps to generate an anonymous face image; wherein the diffusion model comprises a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; an anonymous loss function is introduced into each time step, the cosine similarity between the face image generated in the current time step and the original face image is calculated according to the anonymous loss function, the gradient of the cosine similarity to the face image in the current time step is calculated, and the face image is calculated according to the gradient of the face image in the current time step. And determining the face image generated at the current time step, taking the face image as the input of the next time step, and generating an anonymous face image when anonymous loss calculation of all time steps is completed.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically, to a method, system, device and medium for anonymous face images. Background Art

[0002] Face anonymization refers to the process of processing facial information so that the original identity cannot be identified. The core idea is to use technical means such as de-identification to eliminate or hide sensitive identity information in the image.

[0003] According to different anonymization goals, face anonymization technology can be divided into methods based on visual content modification, methods based on visual content preservation, and visually restorable anonymization methods. Among them, visual content modification means that the visual effect of the image before and after anonymization is different in human eyes; visual content preservation means that the visual appearance of the face before and after anonymization remains unchanged, but the face recognition system believes that the identity of the image before and after anonymization is inconsistent. Visually restorable anonymization methods mean that the anonymized image can be restored to the image before anonymization, and the whole process is "reversible". Traditional face anonymization methods include blurring, mosaics, and occlusion, that is, anonymization is achieved by covering the facial area of ​​the face. With the development of deep generative network technology, face anonymization can generate false faces without affecting the visual quality of the anonymized face while keeping identity-irrelevant attributes such as facial expressions and postures unchanged.

[0004] In the image anonymization method, the related technology can remove the face identity information after anonymization by designing loss functions such as face classification adversarial loss, reconstruction loss, background consistency loss and photo authenticity loss, and restore the original face when the correct password is given, and return a false identity and natural face when an incorrect password is given, so as to achieve reversible anonymity without sacrificing face privacy. However, this method cannot control the anonymity strength of the anonymous face image, which will result in the generated anonymous face image being highly similar to the original face image, and cannot effectively hide the identity sensitive information in the image. Summary of the invention

[0005] The purpose of the present invention is to provide a facial image anonymity method, system, device and medium to solve the problem that related technologies cannot control the anonymity strength of anonymous facial images.

[0006] A first aspect of the present invention provides a method for anonymizing a facial image, the method comprising:

[0007] Acquire an original face image, and extract a coding representation vector of the original face image;

[0008] The encoded representation vector is input into a diffusion model generated by repeated iterations of multiple time steps to generate an anonymous face image; wherein the diffusion model includes a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; an anonymous loss function is also introduced in each time step, and the cosine similarity between the face image generated at the current time step and the original face image is calculated according to the anonymous loss function, and the gradient of the cosine similarity to the face image at the current time step is calculated. According to the gradient of the face image at the current time step, the face image generated at the current time step is determined, and this face image is used as the input of the next time step. When the anonymous loss calculation of all time steps is completed, an anonymous face image is generated.

[0009] In one implementation, a coding representation vector of the original face image is extracted by a downsampling network encoder.

[0010] In one implementation, the reconstruction loss function is expressed as: Where,∈ represents the real noise added by the noise prediction network,∈ θ Represents the noise predicted by the noise prediction model based on the input information, ∈ θ (·) represents the input of the noise prediction model, t represents the noise adding step length, x t Represents the original face image x org According to the noise image obtained by the real noise ∈ and the noise step length t, z org Represents the original face image x org The encoded representation vector extracted by the encoder.

[0011] In one implementation, the anonymous loss function is expressed as: Among them, t represents the current time step, F θ (·) represents the pre-trained face recognition model, x org represents the original face image, represents the face image generated at the current time step, F θ (x org ) represents the identity feature vector of the original face image, Represents the identity feature vector of the face image generated at the current time step.

[0012] In one implementation, the face image generated at the current time step is determined according to the gradient of the face reconstructed image and at the current time step, specifically: a gradient guidance intensity factor representing the identity feature difference between the generated anonymous face image and the original face image is preset;

[0013] The face image generated at the current time step is determined by the difference between the face reconstructed image at the current time step and the product of the gradient guidance strength factor and the gradient.

[0014] In one implementation, the method further includes:

[0015] The original face image is reversely encoded through the diffusion model to extract the random representation vector;

[0016] The encoding representation vector of the anonymous face image is extracted through the downsampling network encoder, and the difference between the encoding representation vector and the original encoding representation vector is calculated to obtain the residual representation vector;

[0017] The random representation vector and the residual representation vector are input into the diffusion model to generate a restored face image.

[0018] A second aspect of the present invention provides a facial image anonymization system, the system comprising:

[0019] A vector extraction module, used to obtain an original face image and extract a coding representation vector of the original face image;

[0020] The image anonymity module is used to input the encoded representation vector into a diffusion model generated by repeated iterations of multiple time steps to generate an anonymous face image; wherein the diffusion model includes a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; an anonymous loss function is also introduced in each time step, and the cosine similarity between the face image generated at the current time step and the original face image is calculated according to the anonymous loss function, and the gradient of the cosine similarity to the face image at the current time step is calculated. According to the gradient of the face image at the current time step, the face image generated at the current time step is determined, and this face image is used as the input of the next time step. When the anonymous loss calculation of all time steps is completed, an anonymous face image is generated.

[0021] In one implementation, the encoding representation vector of the original face image is extracted through a downsampling network of an encoder.

[0022] According to a third aspect of the present invention, an electronic device is provided, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of a facial image anonymity method provided in the first aspect of the present invention are implemented.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of a facial image anonymization method provided in the first aspect of the present invention are implemented.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] In a facial image anonymity method provided by the present invention, the method extracts a coding representation vector of an original facial image, reconstructs a facial reconstructed image based on the coding representation vector, and then combines a diffusion model generated by repeated iterations of multiple time steps, calculates the cosine similarity between the facial reconstructed image of the current time step and the original facial image through an anonymous loss function, calculates the gradient of the cosine similarity to the facial reconstructed image of the current time step, determines the facial image generated at the current time step based on the gradient of the facial reconstructed image and the current time step, and controls the anonymity strength of the anonymous facial image by adjusting the gradient guidance strength factor in the anonymous loss function, so that the anonymous facial image has a lower similarity with the original facial image, thereby effectively hiding identity sensitive information in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of the present invention, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0027] Figure 1 A schematic diagram of a process for anonymizing a facial image provided by an embodiment of the present invention;

[0028] Figure 2 A functional block diagram of a facial image anonymization system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0030] It should be noted that the terms "include" or "may include" used in various embodiments of the present invention indicate the presence of the function, operation or element applied for, and do not limit the addition of one or more functions, operations or elements. In addition, as used in various embodiments of the present invention, the terms "include", "have" and their cognates are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0031] In various embodiments of the present invention, the expression "or" or "at least one of B or / and C" includes any combination or all combinations of the words listed at the same time. For example, the expression "B or C" or "at least one of B or / and C" may include B, may include C, or may include both B and C.

[0032] It should be understood that, in the description of the present invention, “plurality” means two or more than two, unless otherwise clearly and specifically defined.

[0033] Please refer to Figure 1 , Figure 1 A flowchart of a facial image anonymization method provided by an embodiment of the present invention is shown in Figure 1. The method includes:

[0034] S101, obtaining an original face image, and extracting a coding representation vector of the original face image.

[0035] In this embodiment, the original facial image is a collected facial image without any processing. For example, the original facial image can be a facial image collected by a community access control or surveillance camera, or a facial image collected by facial payment software. The original facial image can be collected in real time or non-real time, and this embodiment does not make any special limitations on this.

[0036] Secondly, the original face image can also be cropped and aligned into a standard face image. Specifically, the encoding representation vector of the original face image is extracted through a downsampling network encoder. That is, the encoder uses a U-Net downsampling network to extract high-level semantic information of the face image, and the high-level semantic information is the encoding representation vector.

[0037] S102, input the encoded representation vector into a diffusion model generated by repeated iterations of multiple time steps to generate an anonymous face image; wherein the diffusion model includes a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; an anonymous loss function is also introduced in each time step, and the cosine similarity between the face image generated at the current time step and the original face image is calculated according to the anonymous loss function, and the gradient of the cosine similarity to the face image at the current time step is calculated. According to the gradient of the face image at the current time step, the face image generated at the current time step is determined, and this face image is used as the input of the next time step. When the anonymous loss calculation of all time steps is completed, an anonymous face image is generated.

[0038] In this embodiment, in order to enable the diffusion model to reconstruct the face image according to the input encoding representation vector, this embodiment also introduces a reconstruction loss function in the diffusion model. The reconstruction loss function enables the diffusion model to have the ability to reconstruct the original face image, that is, by updating the parameters of the two neural networks of the encoder and the diffusion model, the loss function used is the reconstruction loss between the original face image and the generated face image, and the expression of the reconstruction loss function is: Where,∈ represents the real noise added by the noise prediction network,∈ θRepresents the noise predicted by the noise prediction model based on the input information, ∈ θ (·) represents the input of the noise prediction model, t represents the noise adding step length, x t Represents the original face image x org According to the noise image obtained by the real noise ∈ and the noise step length t, z org Represents the original face image x org The encoded representation vector extracted by the encoder.

[0039] In a further embodiment, the generation process of the diffusion model is a process of repeated iteration of T time steps, so an anonymous loss function can be introduced in each time step, first calculating the cosine similarity between the identity feature vector of the face image generated at the current time step t and the identity feature vector of the original face image, and then calculating the gradient of the similarity to the face image generated at the time step t, and using the gradient to guide the image generated at the current time step t away from the original face image. Therefore, the combined effect of the pre-trained encoder, the pre-trained generation model, and the anonymization loss function introduced in the generation stage of the generation model, the generated face image can reconstruct the background and face posture of the original image, while changing the facial features, achieving the effect of face anonymization.

[0040] Specifically, the expression of the anonymous loss function is: Among them, t represents the current time step, F θ (·) represents the pre-trained face recognition model, x org represents the original face image, represents the face image generated at the current time step, F θ (x org ) represents the identity feature vector of the original face image, Represents the identity feature vector of the face image generated at the current time step.

[0041] In some embodiments, the face image generated at the current time step is determined based on the face reconstructed image and the gradient of the current time step, specifically: a gradient-guided intensity factor that characterizes the difference in identity features between the generated anonymous face image and the original face image is preset; the face image generated at the current time step is determined by the difference between the face reconstructed image at the current time step and the product of the gradient-guided intensity factor and the gradient.

[0042] Specifically, the cosine similarity between the identity feature vector of the born face image and the identity feature vector of the original face image is calculated through the loss function The similarity is used to calculate the face image generated at the current time step The gradient grad t , the gradient guides the face image generated at the current time step t away from the original face image, and its expressions are:

[0043] Among them, γ represents the gradient guidance strength factor. The setting of γ determines the difference in identity characteristics between the generated anonymous face image and the original face image. When γ is set to a very small value, the guidance strength has a weak effect, and the generated anonymous face image is similar to the original face image; when γ is set to a large value, the guidance strength plays a role, and the generated anonymous face image shows a difference from the original face image. When the similarity is very low, it has an anonymization effect. Therefore, by adjusting the gradient guidance strength factor, the anonymity strength of the anonymous face image can be controlled, so that the anonymous face image has a low similarity with the original face image, thereby effectively hiding the identity-sensitive information in the image.

[0044] In some embodiments, the method also includes: performing reverse encoding processing on the original face image through a diffusion model to extract a random representation vector; extracting the encoding representation vector of the anonymous face image through a downsampling network encoder, and calculating the difference with the original encoding representation vector to obtain a residual representation vector; inputting the random representation vector and the residual representation vector into the diffusion model to generate a restored face image.

[0045] In this embodiment, the encoder and diffusion model have reconstruction capability, also known as restoration capability. In this embodiment, the restoration process requires the provision of specific keys KeyZ and KeyR, where the key KeyZ is the original face image x org The encoded representation vector extracted by the encoder, the key KeyR is composed of the original face image x org The random representation vector extracted by back-encoding the diffusion model.

[0046] Please refer to Figure 2 , Figure 2 A functional block diagram of a facial image anonymization system provided by an embodiment of the present invention, such as Figure 2 As shown, the system includes:

[0047] A vector extraction module 210, used to obtain an original face image and extract a coding representation vector of the original face image;

[0048] The image anonymity module 220 is used to input the encoded representation vector into a diffusion model generated by repeated iterations of multiple time steps to generate an anonymous face image; wherein the diffusion model includes a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; an anonymous loss function is also introduced in each time step, and the cosine similarity between the face image generated at the current time step and the original face image is calculated according to the anonymous loss function, and the gradient of the cosine similarity to the face image at the current time step is calculated. According to the gradient of the face image at the current time step, the face image generated at the current time step is determined, and this face image is used as the input of the next time step. When the anonymous loss calculation of all time steps is completed, an anonymous face image is generated.

[0049] It can be seen that the embodiment of the present invention provides a facial image anonymity system, which extracts the encoding representation vector of the original facial image, reconstructs the facial reconstructed image according to the encoding representation vector, and then combines the diffusion model generated by repeated iterations of multiple time steps, calculates the cosine similarity between the facial reconstructed image of the current time step and the original facial image through the anonymous loss function, calculates the gradient of the cosine similarity to the facial reconstructed image of the current time step, determines the facial image generated at the current time step according to the gradient of the facial reconstructed image and the current time step, and controls the anonymity strength of the anonymous facial image by adjusting the gradient guided strength factor in the anonymous loss function, so that the anonymous facial image has a lower similarity with the original facial image, thereby effectively hiding the identity sensitive information in the image.

[0050] In some embodiments, the encoded representation vector of the original face image is extracted through a downsampling network of an encoder.

[0051] The embodiment of the present application also provides an electronic device, which includes a processor, a memory, a communication interface and at least one communication bus for connecting the processor, the memory and the communication interface. The memory includes but is not limited to a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (PROM) or a portable CD-ROM, and the memory is used for related instructions and data.

[0052] The communication interface is used to receive and send data. The processor may be one or more CPUs. When the processor is a CPU, the CPU may be a single-core CPU or a multi-core CPU. The processor in the electronic device is used to read one or more programs stored in the memory and perform the following operations: obtain the original face image and extract the encoding representation vector of the original face image; input the encoding representation vector into the diffusion model generated by repeated iterations of multiple time steps to generate an anonymous face image; wherein the diffusion model includes a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; an anonymous loss function is also introduced in each time step, and the cosine similarity between the face image generated at the current time step and the original face image is calculated according to the anonymous loss function, and the gradient of the cosine similarity to the face image at the current time step is calculated. According to the gradient of the face image at the current time step, the face image generated at the current time step is determined, and the face image is used as the input of the next time step. When the anonymous loss calculation of all time steps is completed, an anonymous face image is generated.

[0053] It should be noted that the specific implementation of each operation can be Figure 1 The corresponding description of the method embodiment shown, the electronic device can be used to execute a facial image anonymization method of the above method embodiment of the present application, which will not be described in detail here.

[0054] In the embodiments of the present disclosure, a computer-readable storage medium is also provided, and the computer-readable storage medium is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the computer device and the extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, and the storage space stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of a facial image anonymization method in the above embodiment. Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0055] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A facial image anonymization method, characterized in that: Methods include: Acquire an original face image, and extract a coding representation vector of the original face image; The encoded representation vector is input into a diffusion model generated by repeated iterations of multiple time steps to generate an anonymous face image; wherein the diffusion model includes a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; an anonymous loss function is also introduced in each time step, and the cosine similarity between the face image generated at the current time step and the original face image is calculated according to the anonymous loss function, and the gradient of the cosine similarity to the face image at the current time step is calculated. According to the gradient of the face image at the current time step, the face image generated at the current time step is determined, and this face image is used as the input of the next time step. When the anonymous loss calculation of all time steps is completed, an anonymous face image is generated.

2. A facial image anonymization method according to claim 1, characterized in that: The encoding representation vector of the original face image is extracted through a downsampling network encoder.

3. A facial image anonymization method according to claim 1, characterized in that: The expression of the reconstruction loss function is: Where,∈ represents the real noise added by the noise prediction network,∈ θ Represents the noise predicted by the noise prediction model based on the input information, ∈ θ (·) represents the input of the noise prediction model, t represents the noise adding step length, x t Represents the original face image x org According to the noise image obtained by the real noise ∈ and the noise step length t, z org Represents the original face image x org The encoded representation vector extracted by the encoder.

4. A facial image anonymization method according to claim 1, characterized in that: The expression of the anonymous loss function is: Among them, t represents the current time step, F θ (·) represents the pre-trained face recognition model, x org represents the original face image, represents the face image generated at the current time step, F θ (x org ) represents the identity feature vector of the original face image, Represents the identity feature vector of the face image generated at the current time step.

5. The method for anonymizing facial images according to claim 1, characterized in that: According to the gradient of the face reconstructed image and of the current time step, the face image generated at the current time step is determined, specifically: a gradient guidance intensity factor representing the identity feature difference between the generated anonymous face image and the original face image is preset; The face image generated at the current time step is determined by the difference between the face reconstructed image at the current time step and the product of the gradient guidance strength factor and the gradient.

6. A facial image anonymization method according to claim 1, characterized in that: The method further comprises: The original face image is reversely encoded through the diffusion model to extract the random representation vector; The encoding representation vector of the anonymous face image is extracted through the downsampling network encoder, and the difference between the encoding representation vector and the original encoding representation vector is calculated to obtain the residual representation vector; The random representation vector and the residual representation vector are input into the diffusion model to generate a restored face image.

7. A facial image anonymization system, characterized in that: The system includes: A vector extraction module, used to obtain an original face image and extract a coding representation vector of the original face image; The image anonymity module is used to input the encoded representation vector into a diffusion model generated by repeated iterations of multiple time steps to generate an anonymous face image; wherein the diffusion model includes a reconstruction loss function, and the reconstruction loss parameter is the reconstruction loss between the original face image and the face image; an anonymous loss function is also introduced in each time step, and the cosine similarity between the face image generated at the current time step and the original face image is calculated according to the anonymous loss function, and the gradient of the cosine similarity to the face image at the current time step is calculated. According to the gradient of the face image at the current time step, the face image generated at the current time step is determined, and this face image is used as the input of the next time step. When the anonymous loss calculation of all time steps is completed, an anonymous face image is generated.

8. A facial image anonymization system according to claim 7, characterized in that: The encoding representation vector of the original face image is extracted through the downsampling network of the encoder.

9. An electronic device, characterized in that: The electronic device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of a facial image anonymity method as described in any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of a facial image anonymization method according to any one of claims 1 to 6 are implemented.

Images