image anonymization using a synthesized image based on a diffusion algorithm

Abstract

Image anonymization using a synthesized image based on a diffusion algorithm: A computer system obtains a digital source image of a human subject; determines a region to modify within the source image; obtains descriptive text for the source image, including one or more attributes of the human subject represented in the source image; generates a prompt that includes a modified version of the human subject's attribute(s); provides the source image, a representation of the region to modify, and the prompt to a generative AI model (e.g., a text-image model including a diffusion model); and receives a modified image (e.g., an anonymized image) from the generative AI model that includes modified image content (e.g., a generative reconstruction produced by a diffusion model) in the region. The modified image is based on the source image and the modified version of the human subject's attribute(s).Figure for the abridged version: none.

Description

Title of the invention: Image anonymization using a synthesized image based on a diffusion algorithm SUMMARY

[0001] This summary is provided to present a selection of concepts in a simplified form, which are further described below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor to be used as an aid in determining the scope of the claimed subject matter.

[0002] In one aspect, a computer system obtains a digital source image of a human subject; determines a region to be modified in the source image; obtains descriptive text for the source image, including one or more attributes of the human subject represented in the source image; generates a prompt that includes a modified version of the attribute or attributes of the human subject; provides the source image, a representation of the region to be modified, and the prompt to a generative AI model (for example, a text-image model including a broadcast model); and receives a modified image (for example, an anonymized image) from the generative AI model. The modified image is based on the source image and the modified version of the attribute or attributes of the human subject. In some embodiments, the modified image includes a generative reconstruction within the region.Image morphing can also be performed to better match the modified image content with the original image, for example by mixing or realigning features in the modified image content with features in the original image.

[0003] In some embodiments, determining the region to be modified includes the automatic detection of facial features in the digital source image and the automatic calculation of the region to be modified based on the detected facial features. In some embodiments, the region includes one or more facial features of the human subject, such as identifiable personal features (e.g., eyes, nose, mouth, or a combination thereof). The geometry of the region may be based on detected facial features and / or parameters indicating the size, shape, or features to be included in the region to be modified. The computer system may represent the region with an image mask having a geometry that corresponds to the region in the source image.

[0004] In certain embodiments, the computer system obtains the descriptive text for the source image by providing the source image to a pre-trained facial attribute model, extracting one or more attributes using the pre-trained facial attribute model, and generating text based on the attribute or attributes extract(s). The computer system can also obtain one or more contextual attributes from the digital source image, such as a pose attribute or a lighting attribute.

[0005] In some embodiments, the attribute or attributes may include two or more attributes, and the modified version may include modified versions of all the attributes, or the modified version may include at least one unmodified attribute. In some embodiments, prompt generation includes applying a prompt template to the attribute or attributes. The prompt may also include contextual attributes, such as pose or lighting attributes. The prompt may also include additional prompt text or information in addition to specific attributes, which provides further guidance to a generative AI model for generating image content that is not specific to a particular attribute, but may instead affect multiple attributes in the same or different ways.

[0006] In some embodiments, the representation of the region to be modified includes an image mask having geometry that corresponds to the region to be modified in the source image. In some embodiments, image morphing is performed to better match the modified image content with the original image, such as by mixing or realigning features in the modified image content with features in the original image. In some embodiments, the representation of the region to be modified includes an image mask having geometry that corresponds to the region to be modified in the source image.

[0007] Computer-implemented processes, computer-readable media, devices and systems are described. Brief description of the drawings

[0008] The foregoing aspects and many related advantages of this disclosure will be more readily appreciated as they are better understood with reference to the following detailed description, when taken in conjunction with the accompanying drawings, in which:

[0009] [Fig-1] Fig. 1 is a schematic drawing that illustrates an example of a mode of implementation, without limitation, of an image modification system according to various aspects of this disclosure;

[0010] [Fig.2A] The [Fig.2A] is a schematic drawing of processing steps of an algorithm suitable for generating an anonymized image, according to one aspect of this disclosure;

[0011] [Fig.2B] The [Fig.2B] is a schematic drawing of processing steps of an algorithm suitable for generating an anonymized image, according to one aspect of this disclosure;

[0012] [Fig.2C] The [Fig.2C] is a schematic drawing of processing steps of an algorithm suitable for generating an anonymized image, according to one aspect of this disclosure;

[0013] [Fig.2D] The [Fig.2D] is a schematic drawing of processing steps of an algorithm suitable for generating an anonymized image, according to one aspect of this disclosure;

[0014] [Fig.2E] The [Fig.2E] is a schematic drawing of processing steps of an algorithm suitable for generating an anonymized image, according to one aspect of this disclosure;

[0015] [Fig.2F] The [Fig.2F] is a schematic drawing of processing steps of an algorithm suitable for generating an anonymized image, according to one aspect of this disclosure;

[0016] [Fig.3] The [Fig.3] is a schematic diagram of another example of an embodiment of an image modification system according to various aspects of this disclosure;

[0017] [Fig.4] The [Fig.4] is a schematic diagram that illustrates examples of embodiments of a client computer device according to various aspects of this disclosure;

[0018] [Fig.5] The [Fig.5] is a flowchart that illustrates a non-limiting example of an embodiment of a method for obtaining a modified image based on a source image according to various aspects of this disclosure;

[0019] [Fig.6] The [Fig.6] is a schematic diagram that illustrates aspects of an example of a computer device suitable for use as a computer device of this disclosure. Detailed description

[0020] Digital images of human faces are increasingly used in a variety of applications, including beauty apps, entertainment, security, and law enforcement. However, images of living human subjects are often subject to restrictions, including privacy regulations and ethical concerns. In some cases, images can only be used if they are anonymized by altering identifiable personal features within the images. Traditional image anonymization techniques include blurring images or obscuring recognizable facial features (for example, by overlaying black lines over the eye areas). However, these techniques result in the loss of large amounts of image information and make images less useful in many applications.

[0021] Accordingly, this disclosure includes embodiments of novel methods and devices for replacing sensitive facial features in images using generative artificial intelligence (AI) techniques. The described embodiments address the need for privacy-preserving modification of facial attributes while generating photorealistic results.

[0022] In the described embodiments, a computer system incorporates attribute analysis, privacy-focused text construction, and modified image generation functionality. The computer system offers a streamlined process for modifying facial features while prioritizing the user's privacy needs.

[0023] In some embodiments, an input image containing a face is processed using a pre-trained facial attribute model to automatically extract facial features, pose, and skin tone. Additionally, a background attribute model captures contextual attributes of the image. Based on the extracted attributes, descriptive text for the original image is generated, including details such as gender, ethnicity, or age range. To anonymize the image, these details can be replaced with modified descriptive text. The generation of this modified descriptive text can be customized using formulas or algorithms designed to meet specific privacy requirements. The original image can then be provided with a text prompt including the modified descriptive text to a generative AI model, which uses generative AI techniques to generate a modified image.In the modified image, the original selected facial features are replaced by a modified version of those features, the modified version being altered in accordance with the text prompt in a photorealistic manner. In some embodiments, the replacement features are seamlessly integrated into the original image, providing an anonymized but photorealistic version of the original image.

[0024] The described embodiments encompass a wide range of applications in various fields, such as for sensitive tasks in law enforcement or other areas that can benefit from accurate and seamless replacement of facial features and privacy-respecting image manipulation, such as media production, data anonymization, and identity protection. For example, a flexible approach to modifying descriptive text can be used to strike a balance between preserving privacy and achieving accurate reconstruction based on specific application requirements, by example by using modifications that differ more significantly from the original in situations where anonymization is more important.

[0025] Figure 1 is a schematic illustration of a non-limiting example embodiment of an image modification system 100 according to various aspects of this disclosure. In the example shown in Figure 1, a digital source image 90 is provided to a face detection module 110, which detects a face in the source image. The face detection module 110 provides facial feature information to the masking / cropping module 120, which calculates a region in the source image 90 in which corresponding image information (e.g., pixel information) is to be masked or removed (e.g., by cropping). In the example shown in [Fig.1], the masking / cropping module 120 calculates an image mask region 92 which corresponds to a region in the source image 90 to be modified, such as an identifiable personal region (e.g., the eye and nose areas) in the source image 90 which is to be anonymized.The facial analysis module 130 generates descriptive text based on the source image 90, which includes one or more attributes of the image subject. The facial analysis module 130 provides the attribute(s) to the prompt engineering module 140, which generates a prompt including a modified version of the attribute(s). The system 100 provides the modified version of the attribute(s), the source image 90, and the image mask region 92 to the image generation module 150, which generates the modified image 94 based on the source image 90 and the modified version of the attribute(s). In some embodiments, the modified image 94 includes a generative reconstruction within the region corresponding to the image mask 92.In some embodiments, the image generation model 150 includes a generative AI model, which may include a text-image model that uses generative AI techniques to create images based on text and / or image inputs, such as a stable broadcast model, available from Stability AI Ltd., or a Dall-E model, available from OpenAI.

[0026] Figure 2A is a schematic illustration of a non-limiting example of an embodiment of the face detection module 110. In the illustrated embodiment, the face detection module 110 includes a landmark detection module 112 that generates a set of face landmarks in the source image 90, which can be represented as a set of landmark points 114. In some embodiments, the face detection module 110 takes several images as input, such as images taken under different lighting conditions (for example, with different light sources or with light sources in different positions relative to the subject), or from different angles (for example, view of front and profile view), and generates the set of markers based on the analysis of several images.

[0027] Figure 2B is a schematic illustration of a non-limiting example embodiment of the masking / cropping module 120. In the illustrated embodiment, the image mask generator 122 takes the source image 90 and the reference points 114 as input and generates an image mask 124 based on the image mask region 92, which can be overlaid on the source image 90 to restrict image modifications to the specified region. In one illustrative scenario, the image mask generator 122 is configured to calculate an image mask region including areas of the eyes and nose. Alternatively, the image mask generator 122 can be configured to define an image mask corresponding to other regions, such as hair, the mouth, or other features.

[0028] Figure 2C is a schematic illustration of a non-limiting example embodiment of the facial analysis module 130. In the illustrated embodiment, the facial analysis module 130 includes a descriptive text generator 132, which takes the source image 90 as input and generates descriptive text 134 including attributes of the subject of the source image 90. In this example, the attributes include age, race, eye shape, and nose shape. However, more, fewer, or different types of attributes can also be generated, including eye color, gender attributes, complexion attributes (e.g., color), skin texture attributes (e.g., wrinkles, firmness), skin condition attributes (e.g., blemishes, dryness, oiliness, redness), facial landmark attributes (e.g., freckles, moles), and mouth attributes (e.g., size, shape).

[0029] In one embodiment, the descriptive text generator 132 is implemented as a pre-trained facial attribute model, which may also be called an attribute classifier. The pre-trained facial attribute model is configured to extract features such as facial landmarks, color information, and texture information, and to classify images as corresponding to particular attributes (e.g., gender, race, complexion, age, etc.) based on the extracted features, after being trained on face images annotated with known attributes. Different techniques can be used to extract features in particular environments or circumstances. For example, latent variable models can be used to estimate plausible values ​​for occluded or indirectly visible attributes.

[0030] Figure 2D is a schematic illustration of a non-limiting example embodiment of the prompt engineering module 140. In the illustrated embodiment, the descriptive text 134 is converted into a prompt 144 using a prompt template 142. Some attributes in the descriptive text 134 are modified (for example, age is changed from 18 to 30, eye shape from "wide" to "narrow," and nose shape from "flat" to "straight"), while at least one attribute (race) remains unchanged. Modifying these attributes can help ensure confidentiality by allowing the system to generate a modified and anonymized image based on the altered attributes, while preserving image detail to maintain photorealistic quality and retain valuable information. In an illustrative scenario, attributes such as skin texture (e.g., wrinkles, firmness) are preserved even in an anonymized region of the image, to retain such information for, for example, beauty applications such as virtual try-on apps.A prompt containing such modified attributes can be called a confidential encoding prompt. In some embodiments, the modified attributes in the prompt are translated into attribute vectors, which can then be used by the image generation module 150 to generate image data. The extent to which such attributes are modified in the illustrated process can be adjusted in various ways, including settings in a user's privacy profile. For example, a high privacy setting can be associated with larger attribute changes (e.g., a larger age difference in an age attribute), or with changes to a greater number of attributes, than a medium privacy setting. In the example shown in [Fig.[2D], prompt 144 also includes additional text that is not specific to a particular attribute (e.g., "pretty," "young," "natural appearance") which can provide further guidance for a generative AI model to generate image content that is not specific to a particular attribute, but can instead affect multiple attributes in similar or different ways. This additional text can be provided in a prompt template. In an illustrative scenario, a user chooses from several options to modify the source image in particular ways, each option corresponding to a prompt template. Referring to the example in [Fig. 2D], the user can select an option for "pretty, young, natural appearance" or an option associated with a particular celebrity or archetype, and corresponding prompt information can be provided in a corresponding prompt template.Alternatively, the system can automatically select such an option for the user (for example, based on user preferences or profile information).

[0031] Figure 2E is a schematic illustration of a non-limiting example embodiment of the image generation module 150. In the illustrated embodiment, the source image 90, the image mask 124, and the prompt 144 are provided to Text-image model 152, such as a stable diffusion model. In an illustrative implementation, text-image model 152 is implemented as a conditional diffusion model. As shown, text-image model 152 performs a generative reconstruction in the region defined by image mask 124 and generates an intermediate modified image 93 as output. The boundary between the reconstructed area and the original image is represented as a dashed line in the intermediate modified image 93. This illustration represents the possibility that the reconstructed area may not blend seamlessly with the original image. However, this situation can be corrected by performing additional processing on the image to blend the reconstructed area with the original image for better alignment.

[0032] Figure 2F is a schematic illustration of a non-limiting example embodiment of a post-processing module 160, which can be included, for example, in the image generation module 150 or implemented as a separate module in the system 100. In the illustrated embodiment, the image morphing module 162 receives the intermediate modified image 93 as input and mixes and aligns the reconstructed area with the original image data, producing a more photorealistic modified image 94 as output. In an illustrative implementation, the image morphing module 162 calculates a weighted sum of a reconstruction area and the original image and combines these areas in a compositing step, resulting in a more natural-looking modified image but also maintaining the anonymity of the modified image.Additional checks can be performed on a modified image to ensure that the modified image is sufficiently modified, such as comparing vectors representing the modified image and the source image to ensure that the output image is sufficiently different from the source image (e.g., relative to a threshold difference).

[0033] Figure 3 is a schematic illustration of a non-limiting example embodiment of a system 300 that generates anonymized images according to various aspects of this disclosure. In one illustrative scenario, a user captures one or more digital source images of her face with a camera on a client computing device 304 (e.g., a smartphone, tablet computer, laptop, or other computing device). In another illustrative scenario, the user configures the client computing device 304 to capture video or still images using a front-facing camera (also known as a "selfie" camera). Alternatively, the images are captured by a front-facing camera on a laptop or other computing device, or by another image capture device.Adjustments to the source images can be made during the image capture process, including stabilization. electronic or optical image normalization, color correction or similar.

[0034] The client computer device 304 transmits the source image(s) to an image processing computer system 302 via a network 390. The image processing system 302 generates one or more modified versions of the source image(s). In an illustrative scenario, the image processing system 302 retransmits the modified image(s) to the client computer device 304, for example, for display in a user interface, as described in more detail below. Alternatively or in addition, the image processing system 302 stores modified images in an anonymized image data store 318, for example, for model training purposes. In an illustrative scenario, the image processing system 302 stores source images in the source image data store 320, for example, for model training purposes.Alternatively, such as when privacy regulations or user preferences dictate that personally identifiable images cannot be stored in the 302 image processing system, the 302 image processing system discards the source images once they have been modified.

[0035] In the example shown in [Fig. 3], the image processing system 302 includes an image anonymization engine 312, an anonymized image data store 318, and a source image data store 320. In an illustrative implementation, the image anonymization engine 312 includes elements of system 100 (see [Fig. 1]) but uses an external generative AI model such as a text-image model 310 (for example, a broadcast model accessed in a cloud computing arrangement). Alternatively, the text-image model 310 can be implemented within the image processing system 302.

[0036] Figure 4 is a schematic diagram illustrating non-limiting examples of embodiments of a client computing device 304 according to various aspects of this disclosure. As mentioned above, the client computing device 304 transmits image data to the image processing computer system 302 (for example, a desktop computing device, a server computing device, a cloud computing device, or another computing device or combination of computing devices). The client computing device 304 and the computer system 302 can communicate using any suitable communication technology, such as wireless communication technologies, including, but not limited to, Wi-Fi, WiMAX, Bluetooth, 3G, 4G, 5G, and LTE; or wired communication technologies, including, but not limited to, Ethernet, FireWire, and USB.

[0037] In the illustrated embodiment, the client computer device 304 includes a camera 450 and a client application 460, which includes an image preprocessing engine 470, a user interface 476, and a communication module 478. The user interface 476 can present various types of features to a user, such as interactive features like guides, tutorials, or a virtual "try-on" feature for exploring new products or appearances. This technology can, in some embodiments, allow consumers to virtually try on different appearances or products by applying virtual cosmetics to images of their faces. This can be useful in many scenarios, such as creating a color match for cosmetic products to be used in combination, comparing products with alternative products, and so on.This technology can use source images or user-modified images, which can be generated according to the embodiments described herein. In some embodiments, the user interface includes a graphical user interface to assist a user in obtaining high-quality images.

[0038] In some embodiments, image anonymization is used to provide additional user experiences. Consider a scenario in which a user of a virtual try-on application also wants to see how a particular product might look on other people besides themselves. In this situation, it may not be appropriate to use another user's original image, but an image that has been anonymized using the techniques described herein may be useful for this purpose. For example, if the user is searching for products for a family member or friend, the user interface 476 may provide user interface elements that allow the user to select characteristics (e.g., age, eye color, hair color, complexion, etc.) that match the characteristics of the person for whom they are searching for products.In such a situation, the client computer device 304 can transmit the selected attributes to the image processing system 102, which can generate an anonymized image that corresponds to the description by modifying a source image in the source image data store 320. In this scenario, an anonymized image allows the user to perform a virtual try-on of a cosmetic product for another person, without providing an identifiable personal image of another person, thus preserving the confidentiality of the subject of the source image.

[0039] As another example, a user experimenting with different looks or different cosmetic products may wish to share an image of their look to obtain feedback on the appearance (for example, on social networks or a publicly accessible website) without sharing an identifiable image of themselves. In this scenario, the user can upload a self-portrait image to the image processing system 302 for anonymization processing, and the system can transmit a modified version of this image back to the client computer device 304. The user can then be offered options to resynthesize the image or adjust one or more particular attributes to generate a different image (for example, to make the image more anonymous or to experiment with more different appearances).

[0040] In some embodiments, the image preprocessing engine 470 is configured to preprocess images before they are transmitted to the image processing computer system 302 in order to improve the quality of the image editing. In some embodiments, the image preprocessing engine 470 performs image normalization, which may include, for example, color correction, noise reduction or filtering; orientation adjustment; cropping; brightness / exposure adjustment; or contrast adjustment. In an illustrative scenario, an image includes an off-center face that occupies only a small portion of the overall image. To allow for more precise or photorealistic image editing, it may be desirable to reduce the amount of non-face area in the image.This can be accomplished, for example, by using a face detection algorithm to detect the portion of the image that represents the face, centering the face within the image, and zooming in on the image so that the face occupies a larger portion of the image. Other possible normalization actions include cropping the image, reducing or increasing the bit depth, downsampling or upsampling pixels of the image, or similar techniques.

[0041] The product image data can then be sent (potentially with other information, such as a user ID, device ID, or similar) to the communication module 478 for further formatting and transmission to the image processing system 302. (Other features of the client computing device 304 are not shown in Figure 2 for ease of illustration. A description of the illustrative computing devices is given below with reference to [Fig. 6]).

[0042] Numerous alternatives to the arrangements and usage scenarios shown in Figures 1 to 4 are possible. For example, although Figures 3 and 4 illustrate various components as being provided by the client computing device 304 or the image processing computer system 302, in some embodiments the arrangement or functionality of the components may be different. For example, a functionality described as being performed by the client computing device 304 may instead be performed by the image processing computer system 302. or vice versa, or such functionality may be performed by different devices or systems. As another example, the modules and functionalities described with reference to Figures 1 and 2A to 2F may be performed by a computer system including one or more computing devices. As another example, functionality described as being performed by a particular module or component may instead be performed by a combination of such modules or components, or by a different module or component, or functionality described as being performed by individual modules or components may be combined into a single module or component.

[0043] Figure 5 is a flowchart illustrating a non-limiting example of a method 500 for obtaining a modified image (for example, an anonymized version of a face image) based on a source image according to various aspects of this disclosure. The method 500 is performed by a computer system including one or more computing devices. In one embodiment, the method 500 is performed by a computer system such as an image processing computer system 302. Alternatively, the method 500 is performed by another computing device or system.

[0044] In block 502, a computer system obtains a digital source image of a human subject. In block 504, the computer system determines a region to be modified in the source image. In some embodiments, determining the region to be modified includes the automatic detection of facial features in the digital source image and the automatic calculation of the region to be modified based on the detected facial features. The region may include one or more facial features of the human subject, such as identifiable personal features (e.g., eyes, nose, mouth, or a combination thereof). The geometry of the region may be based on detected facial features and / or parameters indicating the size, shape, or features to be included in the region to be modified. The computer system may represent the region with an image mask having a geometry that corresponds to the region in the source image..

[0045] In block 506, the computer system obtains descriptive text for the source image, including one or more attributes of the human subject depicted in the source image. For example, the computer system can obtain the descriptive text for the source image by providing the source image to a pre-trained facial attribute model, extracting one or more attributes using the pre-trained facial attribute model, and generating text based on the extracted attribute(s). The computer system can also obtain one or more contextual attributes from the digital source image, such as a pose attribute or a lighting attribute.

[0046] At block 508, the computer system generates a prompt that includes a modified version of the human subject's attribute or attributes. For example, the one or more attributes may include two or more attributes, and the modified version may include modified versions of all the attributes, or the modified version may include at least one unmodified attribute. The generation of the prompt may include applying a prompt template to the attribute or attributes. The prompt may also include contextual attributes, such as pose or lighting attributes. The prompt may also include additional text or prompt information in addition to specific attributes.Such additional text or prompt information may include, for example, more general prompt information (“natural appearance”; “attractive appearance”; “symmetrical features”; “serious expression”; “playful expression”; “soft features”; “angular features”) that can provide additional guidance for a generative AI model, such as a text-image model, to generate image content that is not specific to a particular attribute, but can instead affect multiple attributes in the same or different ways.

[0047] At block 510, the computer system provides the source image, a representation of the region to be modified, and prompts it to a text-image model, such as a broadcast model. In some embodiments, the representation of the region to be modified includes an image mask having geometry that corresponds to the region to be modified in the source image. At block 512, the computer system receives a modified image (for example, an anonymized image) from the text-image model, including modified image content (for example, a generative reconstruction produced by a broadcast model) in the region corresponding to the image mask. The modified image content is based on the source image and the modified version of the attribute or attributes of the human subject.Image morphing can also be performed to better match the modified image content with the original image, for example by mixing or realigning features in the modified image content with features in the original image.

[0048] Numerous alternatives to the processes shown in Figures 2A to 2F and 5 are possible. For example, the processing steps in the various techniques can be separated into additional steps or combined into fewer steps. As another example, processing steps in the various techniques can be omitted or supplemented by other techniques or processing steps. As another example, processing steps described as occurring in a particular order can instead occur in a different order. As yet another example, processing steps described as being executed in a series of steps can instead be managed in parallel, with multiple modules. or software processes simultaneously managing one or more of the illustrated processing steps. Experiments and experimental results

[0049] 1. Confidentiality and Accuracy#

[0050] Experimental parameters: The GB / T 41772-2022 standard in China for the recovery of old faces requires a FNIR < 10% at FPIR = 0.001. If the generated facial images achieve an FNIR > 10% at FPIR = 0.001 on networks of known faces, the privacy data meets the requirements. Face recovery experiments were conducted using the FaceNet network as the facial recognition model. The experimental parameter was designed to evaluate the performance of the FaceNet model in recovering faces from a generated face database, with the original dataset of individuals serving as the basis truth. The experimental parameter was designed to evaluate the performance of the face retrieval system based on the false negative identification rate (FNIR) at a fixed false positive identification rate (FPIR) of 0.001.

[0051] Face database: A face database was used to conduct the experiments. This face database consisted of natural faces. The database contained a diverse range of facial features, expressions, and poses.

[0052] Facial recognition model: The FaceNet network was used as a facial recognition model. FaceNet is a deep learning model that maps images of faces in a high-dimensional feature space, where the similarity between faces can be measured using the cosine distance.

[0053] Evaluation metric: The performance of the face retrieval system was evaluated on the basis of the False Negative Identification Rate (FNIR) at a fixed False Positive Identification Rate (FPIR) of 0.001. FNIR represents the proportion of true face matches that were incorrectly rejected by the system, while FPIR represents the proportion of imposter face matches falsely identified by the system.

[0054] Expected: To achieve confidentiality objectives such as the dissociability of identities, the undetectability of private attributes, and the selective sharing of attributes according to user preferences. FNIR >10 %@FPIR = 0.001.

[0055] Obtained: Experimental results demonstrated that the FaceNet model achieved a FNIR of 100% at FPIR = 0.001, meaning that the generated images can eliminate identity information. 1. Clinical signs#

[0056] Experimental parameters: To evaluate the differences between the generated faces and the original individuals in terms of 14 signal scores of the NEXA® facial diagnostic algorithm available from Canfield Beauty, we performed the following experimental parameter.

[0057] Data Collection: A diverse dataset of original human faces, including different genders, ages, and ethnic origins, was collected. Each individual's face was captured using high-resolution cameras under controlled lighting conditions. In addition, a set of synthetic faces was generated using a disclosed algorithm and model.

[0058] NEXA® Signal Scoring: The NEXA® facial diagnostic algorithm was used to analyze both original human faces and generated synthetic faces. The NEXA® system provides various signal scores that capture diverse facial scores and clinical signs. These signal scores serve as objective measures to assess the quality and similarity of the generated faces to the original individuals.

[0059] Experimental metric: The mean absolute error (MAE) was used as an evaluation metric to quantify the differences between the signal scores of the generated faces and the original individuals. The MAE measures the mean absolute difference between the predicted signal scores of the generated faces and the basic truth scores of the original individuals on the different dimensions provided by the NEXA® system.

[0060] Experimental procedure: A subset of original human faces was randomly selected from the dataset. For each original face, a corresponding synthetic face was generated using the generative model. The NEXA® system analyzed the original and generated faces, producing signal scores for each face. The EMA between the signal scores of the generated faces and the corresponding original faces was calculated. The experiment was repeated multiple times to ensure its statistical significance.

[0061] Expected: A score difference of less than 0.15 between models trained on synthesized and original photos, which does not indicate a significant difference in score.

[0062] Obtained: The NEXA® scoring algorithm was used to evaluate the score differences. A score difference of 0.12 was obtained between the models trained on synthesized and original photos. Illustrative computer environments

[0063] In general, the word "engine", as used here, refers to logic embedded in hardware or software instructions, which can be written in a A programming language, such as C, C++, COBOL, Java™, PHP, Perl, H™L, CSS, JavaScript, VBScript, ASPX, Microsoft .NET™, and / or similar. An engine can be compiled into executable programs or written in interpreted programming languages. Software engines can be called from other engines or from themselves. In general, the engines described here refer to logical modules that can be merged with other engines or can be divided into sub-engines. Engines can be stored on any type of computer-readable media or computer storage device and can be stored on one or more general-purpose computers and executed by them, thus creating a specialized computer configured to provide the engine or its functionality.

[0064] As understood by a person skilled in the art, a "data store" as described herein can be any suitable device configured to store data for access by a computing device. An example of a data store is a high-throughput, highly reliable relational database management system (DBMS) running on one or more computing devices and accessible over a high-speed network. Another example of a data store is a key-value store. However, any other suitable storage technique and / or device capable of quickly and reliably providing the stored data in response to queries can be used, and the computing device can be accessed locally or provided as a cloud service.A data store may also include data stored in an organized manner on a computer-readable storage medium, as described below. A person skilled in the art will recognize that separate data stores described herein may be combined into a single data store, and / or that a single data store described herein may be separated into multiple data stores, without departing from the scope of this disclosure.

[0065] Figure 6 is a functional diagram that illustrates aspects of an example computing device suitable for use as a computing device in this disclosure. Although many different types of computing devices have been discussed above, the example computing device describes various features common to many different types of computing devices. Although Figure 6 is described with reference to a computing device that is implemented as a device on a network, the description below is applicable to servers, personal computers, mobile phones, smartphones, tablets, embedded computing devices, and other devices that may be used to implement portions of embodiments of this disclosure. Furthermore, a person skilled in the art and others will recognize that the Computer device 600 can be any one of a number of devices currently available or yet to be developed.

[0066] In its most basic configuration, the computing device 500 includes at least one processor 602 and a system memory 604 connected by a communication bus 606. Depending on the exact configuration and type of device, the system memory 604 may be volatile or non-volatile memory, such as read-only memory (“ROM”), random-access memory (“RAM”), EEPROM, flash memory, or similar memory technology. Those skilled in the art and others will recognize that the system memory 604 typically stores data and / or program modules that are immediately accessible to the processor 602 and / or are currently being used by it. In this respect, the processor 602 can serve as the computing center of the computing device 600 by handling the execution of instructions.

[0067] As illustrated in more detail in [Fig. 6], the computer device 600 may include a network interface 610 comprising one or more components for communicating with other devices on a network. Embodiments of this disclosure may access basic services that use the network interface 610 to perform communications using common network protocols. The network interface 610 may also include a wireless network interface configured to communicate via one or more wireless communication protocols, such as Wi-Fi, 3G, 4G, 5G, LTE, WiMAX, Bluetooth, Bluetooth Low Energy, and / or similar protocols. As a person skilled in the art will understand, the network interface 610 illustrated in [Fig. 6] may represent one or more of the wireless or physical communication interfaces described and illustrated above.

[0068] In the example embodiment shown in [Fig. 6], the computer device 600 also includes a storage medium 608. However, it is possible to access the services using a computer device that does not include means for data persistence on a local storage medium. Therefore, the storage medium 608 shown in [Fig. 6] is represented by a dashed line to indicate that the storage medium 608 is optional. In any case, the storage medium 508 may be volatile or non-volatile, removable or non-removable, implemented using any technology capable of storing information such as, but not limited to, a hard disk drive, an SSD, a CD-ROM, a DVD or any other disk storage, magnetic cassettes, magnetic tape, magnetic disk storage and / or the like.

[0069] As used herein, the term "computer-readable medium" includes volatile and non-volatile, removable and non-removable media implemented in any process or technology capable of storing information, such as instructions Computer-readable data structures, program modules, or other data. In this regard, system memory 604 and storage medium 608 shown in [Fig. 6] are only examples of computer-readable media.

[0070] Suitable implementations of computing devices including a processor 602, system memory 604, a communication bus 606, a storage medium 608, and a network interface 610 are known and commercially available. For ease of illustration and because it is not important for understanding the claimed subject matter, [Fig. 6] does not show some of the typical components of many computing devices. In this regard, the computing device 600 may include input devices, such as a keyboard, numeric keypad, mouse, microphone, touchscreen, touchpad, and / or the like. These input devices may be coupled to the computing device 600 by wired or wireless connections, including RF, infrared, serial, parallel, Bluetooth, Bluetooth Low Energy, USB, or other suitable connection protocols using wireless or physical connections.Similarly, the 600 computer device may also include output devices such as a display, speakers, a printer, etc. Since these devices are well-known in the arts, they are not illustrated or described in further detail here.

[0071] Although illustrative embodiments have been shown and described, it will be appreciated that various changes can be made to them without departing from the spirit and scope of the invention.

Claims

Demands

1. A computer-implemented method for obtaining a modified image based on a digital source image, the method comprising: - obtaining a digital source image of a human subject; - determining a region to be modified in the digital source image; - obtaining descriptive text for the digital source image, in which the descriptive text includes one or more attributes of the human subject represented in the digital source image; - generating a prompt that includes a modified version of one or more attributes of the human subject; - providing the digital source image, a representation of the region to be modified, and said prompt to a generative AI model; - receiving a modified image from the generative AI model, in which the modified image is based on the digital source image and the modified version of one or more attributes of the human subject.

2. A method according to claim 1, wherein the generative AI model is a text-image model comprising a diffusion model, the modified image comprising a generative reconstruction within the region.

3. A method according to any one of claims 1 and 2, wherein the representation of the region to be modified comprises an image mask having a geometry that corresponds to the region in the digital source image.

4. A method according to any one of claims 1 to 3, wherein the one or more attributes include one or more of a gender attribute, a race attribute, an age attribute, an eye attribute, a nose attribute, a complexion attribute, a skin texture attribute, a skin condition attribute, a facial landmark attribute, a hair attribute, a mouth attribute.

5. A method according to any one of claims 1 to 4, wherein the region includes one or more facial features of the human subject, preferably said one or more facial features include the eyes and nose of the human subject.

6. A method according to any one of claims 1 to 5, wherein obtaining the descriptive text comprises: - providing the digital source image to a pre-trained facial attribute model; and - extracting one or more attributes using the pre-trained facial attribute model.

7. A method according to any one of claims 1 to 6, wherein the prompt generation includes applying a prompt template to one or more attributes.

8. A method according to any one of claims 1 to 7, further comprising obtaining one or more contextual attributes from the digital source image, wherein the modified image content is further based on the one or more contextual attributes, preferably said one or more contextual attributes include a pose attribute or a lighting attribute.

9. Non-transient, computer-readable medium on which are stored computer-executable instructions configured to cause a computer system to carry out the method of any one of claims 1 to 8.

10. Computer system comprising at least one processor and a memory, memory in which are stored computer-executable instructions configured to cause the computer system to execute the method of any one of claims 1 to 8.

Images