Image processing and identification photo generation method and device, storage medium and program product

Abstract

Embodiments of the present application provide an image processing and ID photo generation method, device, storage medium and program product. In the embodiments of the present application, the gender attribute of the user can be recognized according to the initial facial feature information contained in the face image; the gender attribute of the user corresponds to a plurality of ID photo templates, and a target ID photo template can be selected from the plurality of ID photo templates; further, the initial facial feature information can be fused into the target ID photo template by using an image fusion model, so as to obtain a first intermediate ID photo; further, the image repair model is used to repair the face region in the first intermediate ID photo to obtain a second intermediate ID photo, so that the face region of the ID photo is clearer. Then, the text-to-image model can be used to finely process the second intermediate ID photo according to at least one prompt word to obtain a target ID photo, so as to improve the clarity of the target ID photo and the overall similarity with the original photo.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing and ID photo generation method and device, a storage medium and a program product. BACKGROUND

[0002] In daily life, learning and work, various types of photos are often needed to be taken, such as ID photos required for people to handle work cards, student cards, driver's licenses, wedding photos, etc. For the same type of ID photo, there are many photo styles and certain size, color, instrument requirements, and professional photographers are usually needed to collect photos, but it is not convenient for professional photographers to collect photos.

[0003] With the continuous development of image processing technology, various photographing applications have emerged. At present, most of the photographing applications on the market generate photos based on generative algorithms. Specifically, multiple face images of a user are input, and an image processing model is used to process the multiple face images, and the processed face images are pasted onto an ID photo template to obtain an ID photo.

[0004] However, the type of ID photo generated by the above method is single, and the clarity and overall similarity with the original photo are low. SUMMARY

[0005] The present application provides an image processing and ID photo generation method, device, storage medium and program product to improve the clarity of the generated photo and the overall similarity with the original photo.

[0006] The present application provides an ID photo generation method, comprising: obtaining a face image of a user, the face image containing initial facial feature information of the user; identifying a gender attribute of the user according to the initial facial feature information contained in the face image; selecting a target ID photo template according to the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user; inputting the initial facial feature information and the target ID photo template into an image fusion model, fusing the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo; inputting the first intermediate ID photo into an image repair model, repairing a face region in the first intermediate ID photo to obtain a second intermediate ID photo; inputting the second intermediate ID photo and at least one prompt word corresponding to the target ID photo template into a text-to-image model, and performing fine processing on the second intermediate ID photo according to the at least one prompt word to obtain a target ID photo.

[0007] The embodiment of the present application further provides an image processing method, comprising: obtaining a face image of a target object, the face image containing initial facial feature information of the target object; identifying a gender attribute of the target object according to the initial facial feature information contained in the face image; selecting a target photo template according to the gender attribute of the target object and a plurality of photo templates corresponding to the gender attribute of the target object; inputting the initial facial feature information and the target photo template into an image fusion model, fusing the initial facial feature information into the target photo template to obtain a first intermediate photo; inputting the first intermediate photo into an image repairing model, repairing a face region in the first intermediate photo to obtain a second intermediate photo; inputting the second intermediate photo and at least one prompt word corresponding to the target photo template into a text-to-image model, and performing fine processing on the second intermediate photo according to the at least one prompt word to obtain a target photo.

[0008] The embodiment of the present application further provides an electronic device, comprising: a memory and a processor; the memory is used for storing a computer program; the processor is coupled with the memory and is used for executing the computer program in the memory to implement the steps in the ID photo generation method or the image processing method provided by the embodiment of the present application.

[0009] The embodiment of the present application further provides a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor can implement the steps in the ID photo generation method or the image processing method provided by the embodiment of the present application.

[0010] The embodiment of the present application further provides a computer program product, comprising a computer program / instruction, when the computer program / instruction is executed by a processor, the processor can implement the steps in the ID photo generation method or the image processing method.

[0011] The technical scheme provided by the embodiment of the application, on the one hand, for the generation of a user's ID photo, a plurality of image models can be used for multi-stage image processing according to the face image of the user, specifically, the gender attribute of the user is recognized according to the initial facial feature information contained in the face image; the gender attribute of the user corresponds to a plurality of ID photo templates, so that a target ID photo template is selected from the plurality of ID photo templates; further, the initial facial feature information can be fused into the target ID photo template by using an image fusion model, so as to obtain a first intermediate ID photo; further, the face region in the first intermediate ID photo is repaired by using an image repair model to obtain a second intermediate ID photo, so that the face region of the ID photo is clearer. After obtaining the clear second intermediate ID photo, the second intermediate ID photo can be finely processed according to at least one prompt word by using a text-to-image model, so as to obtain a target ID photo. Therefore, a high-definition target ID photo can be obtained without collecting multiple face images of the user before image processing, and the overall similarity between the target ID photo and the original photo can also be improved.

[0012] On the other hand, for the generation of various types of photos of a target object, multi-stage image processing can be performed according to the face image of the target object by using a plurality of image models, specifically, the gender attribute of the target object is recognized according to the initial facial feature information contained in the face image; the gender attribute of the target object corresponds to a plurality of photo templates, so that a target photo template is selected from the plurality of photo templates; further, the initial facial feature information can be fused into the target photo template by using an image fusion model, so as to obtain a first intermediate photo; further, the face region in the first intermediate photo is repaired by using an image repair model to obtain a second intermediate photo, so that the face region of the photo is clearer. After obtaining the clear second intermediate photo, the second intermediate photo can be finely processed according to at least one prompt word by using a text-to-image model, so as to obtain a target photo. Therefore, a high-definition target photo can be obtained without collecting multiple face images of the target object before image processing, and the overall similarity between the target photo and the original photo can also be improved. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings, which are included to provide a further understanding of the application, constitute a part of this application, the illustrative embodiments of the application and their description serve to explain the application, and do not constitute an improper limitation on the application. In the drawings:

[0014] Figure 1a The flowchart of the ID photo generation method provided by an exemplary embodiment of the application is shown in the figure;

[0015] Figure 1b The contrast schematic diagram of the original face image and the optimized ID photo provided by an exemplary embodiment of the application is shown in the figure;

[0016] Figure 2 A flowchart of an image processing method provided for an exemplary embodiment of the present application is shown in FIG. 1.

[0017] Figure 3 A structural diagram of a photo generating device provided for an exemplary embodiment of the present application is shown in FIG. 2.

[0018] Figure 4 A structural diagram of an image processing device provided for an exemplary embodiment of the present application is shown in FIG. 3.

[0019] Figure 5 A structural diagram of an electronic device provided for an exemplary embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be described below in detail with reference to the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0021] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use, and processing of related data need to comply with relevant laws, regulations, and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal. In addition, the various models (including but not limited to language models or large models) involved in the present application are in compliance with relevant laws and standard regulations.

[0022] At present, most of the photographing software on the market generates photos based on generative algorithms. Specifically, multiple original photos of a face are input, and an image processing model is used to process the multiple original photos of the face to obtain a photo of a type desired by the user. However, the photo generated by this method has certain requirements for the input original photo, such as the need to input multiple original photos, the original photos must be front-view photos, and the resolution and facial clarity cannot be too low, and the type of the generated ID photo is single; in addition, the overall similarity between the generated ID photo and the original photo is not high.

[0023] To solve the above technical problems, in the embodiments of the present application, for the generation of a user's ID photo, a plurality of image models can be used for multi-stage image processing according to the user's face image. Specifically, the gender attribute of the user is identified according to the initial facial feature information contained in the face image. The gender attribute of the user corresponds to a plurality of ID photo templates, and thus a target ID photo template is selected from the plurality of ID photo templates. Further, the initial facial feature information can be fused into the target ID photo template by using an image fusion model, so as to obtain a first intermediate ID photo. Further, the face region in the first intermediate ID photo is repaired by using an image repair model to obtain a second intermediate ID photo, so that the face region of the ID photo is clearer. Then, in the case of obtaining the clear second intermediate ID photo, the second intermediate ID photo can be finely processed according to at least one prompt word by using a text-to-image model, so as to obtain a target ID photo. Therefore, without collecting multiple face images of the user before image processing, a target ID photo with high clarity can be obtained, and the overall similarity between the target ID photo and the original photo can also be improved.

[0024] On the other hand, for the generation of various types of photos of a target object, a plurality of image models can be used for multi-stage image processing according to the face image of the target object. Specifically, the gender attribute of the target object is identified according to the initial facial feature information contained in the face image. The gender attribute of the target object corresponds to a plurality of photo templates, and thus a target photo template is selected from the plurality of photo templates. Further, the initial facial feature information can be fused into the target photo template by using an image fusion model, so as to obtain a first intermediate photo. Further, the face region in the first intermediate photo is repaired by using an image repair model to obtain a second intermediate photo, so that the face region of the photo is clearer. Then, in the case of obtaining the clear second intermediate photo, the second intermediate photo can be finely processed according to at least one prompt word by using a text-to-image model, so as to obtain a target photo. Therefore, without collecting multiple face images of the target object before image processing, a target photo with high clarity can be obtained, and the overall similarity between the target photo and the original photo can also be improved.

[0025] The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0026] Figure 1b A flowchart of an ID photo generation method provided by an exemplary embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the method includes the following steps. Figure 1b

[0027] 101, obtaining a face image of a user, the face image containing initial facial feature information of the user;

[0028] ​102. Identify the gender attribute of the user according to the initial facial feature information contained in the face image;

[0029] 103. Select a target ID photo template according to the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user;

[0030] 104. Input the initial facial feature information and the target ID photo template into an image fusion model, fuse the initial facial feature information into the target ID photo template, to obtain a first intermediate ID photo;

[0031] 105. Input the first intermediate ID photo into an image inpainting model, inpaint the face region in the first intermediate ID photo, to obtain a second intermediate ID photo;

[0032] 106. Input at least one prompt word corresponding to the target ID photo template into a text-to-image model, and perform fine processing on the second intermediate ID photo according to the at least one prompt word, to obtain the target ID photo.

[0033] In the embodiments of the present application, the user can be a person engaged in different posts who needs to generate an ID photo. The embodiments do not limit the types of posts. For example, the post can be a chef post, a lawyer post, an administrative staff post, or a housekeeping staff post, etc. Different posts have different requirements for ID photos. For the chef post, the requirement for the ID photo can be to wear a chef hat; for the lawyer post, the requirement for the ID photo can be to wear a suit and tie; for the administrative staff post, the requirement for the ID photo can be to wear a bun or a short haircut; for the housekeeping staff post, the requirement for the ID photo is to wear housekeeping clothes, etc., which are not limited in the embodiments. Moreover, different recruitment companies can have different requirements for ID photos for the same post. In addition, different posts or different recruitment companies can also have different requirements for the background color of the ID photo. For example, the background color can be white, blue, or red, etc.

[0034] In the embodiments of the present application, a plurality of types of ID photo templates can be provided, and for the same type of ID photo, a plurality of templates can also be provided for the user to select, so as to improve the diversity of the ID photo.

[0035] Further, in order to enable the user to accurately select the ID photo template, the embodiment can also divide the gender attribute of the user, and provide a plurality of ID photo templates for users with different gender attributes, so that the user can select an ID photo template meeting the user's needs. Specifically, a face image of the user can be acquired, and the face image contains initial facial feature information of the user. Further, the gender attribute of the user is identified according to the initial facial feature information contained in the face image. Then, the target ID photo is selected according to the gender attribute of the user and the plurality of ID photo templates corresponding to the gender attribute of the user. The facial features include at least the user's eyebrows, eyes, nose, mouth and ears. The initial facial feature information includes at least the feature information of the user's eyebrows, eyes, nose, mouth and ears contained in the face image, and the feature information specifically refers to the size, position, color and shape of the above-mentioned facial features.

[0036] In the embodiment, the way of acquiring the face image of the user is not limited. For example, the camera of the terminal device on which the photographing application is installed can be called to capture the face image of the user in real time when generating the ID photo; or the pre-captured face image of the user can be selected from the album of the terminal device on which the photographing application is installed; or the pre-captured face image of the user can be acquired from other terminal devices or databases, etc. In addition, the posture of the user when capturing the face image is not limited in the embodiment, as long as the face image contains the facial feature information of the user. For example, it can be a face image in a sitting posture, a face image in an inclined posture or a face image in a lying posture, etc. Furthermore, the number of face images of the user acquired in the embodiment is not limited, as long as the face image contains the facial feature information of the user. For example, only one face image of the user can be acquired, and the ID photo is generated based on the one face image; or a plurality of face images of the user can be acquired, and the ID photo is generated based on the plurality of face images, etc. In the embodiment, one face image of the user is mainly acquired, so that the number of ID photos used can be reduced, and the workload of capturing the face image can be reduced.

[0037] In an optional embodiment, the initial facial feature information of the user can be directly extracted from the facial image of the user, and the gender attribute of the user is identified according to the initial facial feature information. Specifically, the gender attribute of the user is identified according to the initial facial feature information contained in the facial image, including: inputting the target facial image into the feature extraction network layer in the face recognition model to extract the initial facial feature, to obtain the initial facial feature information; inputting the initial facial feature information into the gender classification network layer of the face recognition model to perform gender classification, to obtain the gender attribute of the user. The type of the face recognition model is not limited in this embodiment. For example, the face recognition model can be an Insight Face model. Insight Face is a face recognition system based on deep learning, which has the characteristics of high efficiency, accuracy and scalability. It can use advanced neural network technology to identify and verify faces in various complex environments.

[0038] Optionally, the target facial image is input into the feature extraction network layer in the face recognition model to extract the facial feature, to obtain the facial feature information, including: first extracting the primary features of the image, such as the edges and textures of the facial features; gradually combining the primary features into more complex feature representations, which contain the initial facial feature information; extracting the features according to the required feature level to obtain the facial feature information.

[0039] In another optional embodiment, the semantic segmentation model can also be used to perform semantic segmentation on the facial image, and the gender attribute of the user is identified according to the initial facial feature information contained in the local image after semantic segmentation. Through the semantic segmentation model, a more accurate local image containing the initial facial feature information can be segmented, and the initial facial feature information is extracted based on the accurate local image, which can reduce the data processing amount on the one hand, and improve the accuracy of the extracted initial facial feature information on the other hand.

[0040] The semantic segmentation refers to a technology in the field of computer vision and image processing, and aims to assign each pixel in an image to a specific category or object, so as to realize identification and segmentation of all objects in the image, such as identification and segmentation of five facial feature objects including eyebrows, eyes, nose, mouth and ears. The semantic segmentation model is obtained by pre-training based on a plurality of face image samples containing five facial feature information. The embodiment does not limit the type of semantic segmentation model. For example, the semantic segmentation model can be a segmentation model based on a convolutional neural network (CNN) deep learning, a U-Net model, a DeepLab series model or a MobileNetV2 model, and the like. The DeepLab series model includes but is not limited to DeepLabv1, DeepLabv2, DeepLabv3 and DeepLabv3+ models. The semantic segmentation model can also be a model based on a Vision Transformer (ViT) type, such as a SETR (Semantic Segmentation Transformer) model, a Segmenter model or a SegFormer model, and the like.

[0041] In this another optional embodiment, according to the initial five facial feature information contained in the face image, the gender attribute of the user is identified, including: inputting the face image into the semantic segmentation model for semantic segmentation to obtain a binary image containing the five facial feature regions; obtaining a local image of the five facial feature regions according to the target face image and the binary image, the local image being part of the target face image; inputting the local image into a feature extraction network layer in the face recognition model to extract the initial five facial feature information of the local image; and inputting the initial five facial feature information of the local image into a gender classification network layer in the face recognition model to classify the gender, so as to obtain the gender attribute of the user.

[0042] The binary image of the facial feature region refers to a two-dimensional array or image with the same size as the facial feature region, which visually presents a black and white image. Each pixel value in the binary image represents the belonging or attribute of the pixel in a certain classification or identification task. Specifically, the binary image can be used to represent the result of image segmentation, in which each pixel is assigned a probability label representing the likelihood of the corresponding pixel belonging to a certain class. The class with the highest probability is usually used to determine the final class label of each pixel.

[0043] Optionally, the face image is input into a semantic segmentation model for semantic segmentation to obtain a binary image containing the facial feature regions, including: inputting the face image into an encoder of the semantic segmentation model, and gradually extracting facial feature regions using the encoder; inputting the facial feature regions into a decoder of the semantic segmentation model, and gradually restoring the image resolution using the decoder to obtain pixel values of each pixel; based on the pixel reference values of each facial feature region and the pixel values of each pixel, calculating the class probability of each facial feature region to which each pixel belongs to obtain a probability map of each pixel in each facial feature region; and converting the probability map into a binary image, in which the facial feature regions are marked as 1 and other regions are marked as 0. Each pixel is assigned a probability representing the likelihood of the pixel belonging to a certain class, and the class with the highest probability is usually used to determine the final class label of each pixel.

[0044] More specifically, in the encoder part, the successive convolution and pooling operations of the semantic segmentation model gradually reduce the spatial dimensions of the face image while increasing the depth of the facial feature to capture more abstract facial features, i.e., high-level abstract features. High-level abstract features refer to complex and deep-level feature representations extracted from raw data in machine learning and computer vision. These features are not just simple pixel values or basic image attributes, but rather high-level interpretations and understandings of data that can capture deeper patterns and information in the data. In addition, high-level abstract features have the following characteristics: hierarchy, high-level abstract features are usually at a higher level in the hierarchy of feature extraction, which is based on the abstraction and combination of low-level features such as edges, colors, and textures; complexity, high-level abstract features are more complex than low-level features and can represent more complex data patterns, such as parts, shapes, or overall structures of a face; semantics, high-level abstract features often have stronger semantics, i.e., they can better represent the semantic information of data, such as facial expressions, facial feature categories, etc.; invariance, high-level abstract features usually have certain invariance, such as robustness to image transformations such as translation, rotation, and scaling; multi-scale, high-level abstract features may contain information at different scales, capable of capturing both local details and global context; cross-domain, some high-level abstract features may be applicable in different domains or tasks, for example, features learned from natural images may also be applicable to medical image analysis; feature fusion, in some cases, high-level abstract features may be the result of the fusion of features of different types or sources to obtain a more comprehensive representation.

[0045] In the decoder part, the semantic segmentation model gradually recovers the spatial dimensions of the image through upsampling and transposed convolution, while reducing the depth of the facial feature, to obtain a feature map of the facial feature; at the end of the decoder, the feature map is converted into the class probability of each pixel through one or more convolution layers, which usually involves a convolution layer with multiple output channels, each channel corresponding to a class probability; the output of the convolution layer is converted into a probability distribution using the Softmax or other activation function, and the Softmax function can ensure that the sum of the probabilities of all classes for each pixel is 1; a probability map is output, where each element corresponds to the pixel at the corresponding position in the input face image and contains the probability that the pixel belongs to each class; each pixel in the probability map is binarized according to a determined threshold to obtain a binary image. If the probability of a pixel belonging to a certain class is higher than the threshold, the pixel is marked as 1 (or the corresponding class label) in the binary image, otherwise it is marked as 0. The threshold usually refers to a global threshold used to convert continuous probabilities into discrete class labels. For all pixels in the same image, the threshold is the same. This is because the threshold is a global processing method that determines the class membership based on the probability distribution of the entire image, rather than setting a threshold for each pixel individually. The specific value of the threshold is pre-set according to actual needs, and the threshold can be fixed or dynamically selected based on the performance on the validation set.

[0046] It should be noted that the binary image of the facial feature region in the embodiment can be one binary image containing the facial feature region; or each facial feature can correspond to one binary image, and the binary image of the face includes multiple binary images as follows: a binary image corresponding to the eyebrow region, a binary image corresponding to the eye region, a binary image corresponding to the nose region, a binary image corresponding to the mouth region, and a binary image corresponding to the ear region.

[0047] Optionally, according to the target face image and the binary image, a local image of the facial feature region is obtained, including: adjusting the face image to the same size and direction as the binary image based on the size of the binary image and the contained face region, to ensure that the two are aligned; locating the position of the facial feature region according to the binary image; and cropping the face image according to the positioning result to obtain a facial feature local image corresponding to the marked region in the binary image. It should be noted that when the position of the facial feature region is located according to the binary image, the non-zero values (representing the facial feature region) in the binary image can be scanned to achieve this. It should be noted that if the facial feature region is close to the edge of the image, special processing may be required during cropping to ensure the integrity of the local image.

[0048] Optionally, the facial feature information of the local image is input into a feature extraction network layer in the face recognition model to extract facial feature information, including: first extracting primary features of the local image, such as edges and textures of the facial features; gradually combining the primary features into more complex feature representations, which contain initial facial feature information; extracting features according to the required feature level to obtain the facial feature information.

[0049] Optionally, the facial feature information of the local image is input into a gender classification network layer in the face recognition model to perform gender classification, to obtain the gender attribute of the user, including: performing dimension reduction processing on the initial facial feature information to obtain a low-dimensional feature vector of the initial facial feature information; performing gender recognition on the low-dimensional feature vector of the initial facial feature information through the gender classification network layer to obtain the gender attribute of the user. The gender classification network layer of the face recognition model is obtained by pre-training based on a set of facial image sample information and the gender of the corresponding facial image samples. The training process of the gender classification network layer of the face recognition model is as follows: obtaining a set of facial image sample information and the gender of the corresponding image samples; determining the key points of each sample in the set of facial image sample information, the key points being the corner positions of each facial feature; obtaining the initial facial feature information of each sample in the set of facial image sample information according to each sample and its key points; performing dimension reduction processing on the initial facial feature information of each sample in the set of facial image sample information to obtain a low-dimensional feature vector of each sample; training the face recognition model through the low-dimensional feature vector of each sample and its corresponding gender value to obtain the face recognition model. More specifically, when training the face recognition model, the gender classification network layer of the face recognition model is focused on to obtain a gender classification network layer with gender recognition function.

[0050] The dimension reduction processing on the initial facial feature information to obtain a low-dimensional feature vector of the initial facial feature information includes: obtaining a feature dimension reduction matrix and the initial facial feature information; performing dimension reduction processing on the initial facial feature information through the feature dimension reduction matrix to obtain a low-dimensional feature vector of the initial facial feature information.

[0051] The gender recognition network layer of the face recognition model can include a standard female face network layer, a standard male face network layer, and a similarity measurement network layer. Based on this, in an optional embodiment, the low-dimensional feature vector of the initial facial feature information is subjected to gender recognition through the gender classification network layer, including: presetting a first similarity threshold of a standard female face model and a second similarity threshold of a standard male face model; obtaining, through a similarity measurement model, a first similarity between the low-dimensional feature vector of the initial facial feature information and the standard female face model and a second similarity between the low-dimensional feature vector of the initial facial feature information and the standard male face model; determining whether the first similarity exceeds the first threshold; if the first similarity exceeds the first threshold, the initial facial feature information is preliminarily determined to be female; otherwise, the initial facial feature information is preliminarily determined to be male; determining whether the second similarity exceeds the second threshold; if the second similarity exceeds the second threshold, the initial facial feature information is preliminarily determined to be male; otherwise, the initial facial feature information is preliminarily determined to be female; if the determination is consistent, the determination result is output; if the determination is inconsistent, an unrecognizable result is output. The gender recognition process in the double-verification mode can improve the accuracy of the gender recognition result.

[0052] In another optional embodiment, a gender recognition process in a recognition mode can be used. The low-dimensional feature vector of the initial facial feature information is subjected to gender recognition through the gender classification network layer, including: presetting a similarity difference threshold; obtaining, through a similarity measurement model, a first similarity between the low-dimensional feature vector of the initial facial feature information and the standard female face model and a second similarity between the low-dimensional feature vector of the initial facial feature information and the standard male face model; determining whether the absolute value of the difference between the first similarity and the second similarity is less than the similarity difference threshold; if the absolute value is less than the similarity difference threshold, an unrecognizable result is output; if the absolute value is greater than the similarity difference threshold, the gender type corresponding to the greater value of the similarity is output.

[0053] In an optional embodiment, after the gender attribute of the user is recognized, the target ID photo template is selected according to the gender attribute of the user and the plurality of ID photo templates corresponding to the gender attribute of the user, including: selecting, according to the gender attribute of the user, the plurality of ID photo templates corresponding to the gender attribute of the user, and sending the plurality of ID photo templates to the terminal device of the user, so that the terminal device displays the plurality of ID photo templates to the user; and receiving a selection request sent by the terminal device in response to a selection operation of the user, and determining any ID photo template selected by the user as the target ID photo template according to the selection request.

[0054] In another optional embodiment, the target ID photo template is selected according to the gender attribute of the user and the plurality of ID photo templates corresponding to the gender attribute of the user, comprising: obtaining the template requirement information of the user; determining the plurality of ID photo templates corresponding to the gender attribute of the user according to the gender attribute of the user; selecting an ID photo template that matches the template requirement information of the user from the plurality of ID photo templates as the target ID photo template.

[0055] Further, after selecting the target ID photo template, the initial facial feature information of the user and the target ID photo can be fused through an image fusion model to obtain a fused ID photo. In order to facilitate description and differentiation, the fused ID photo is referred to as a first intermediate ID photo. The image fusion model includes a style adaptation network layer and a feature merging network layer. Based on this, the initial facial feature information and the target ID photo template are input into the image fusion model, and the initial facial feature information is fused into the target ID photo template to obtain the first intermediate ID photo, comprising: inputting the initial facial feature information and the target ID photo template into the style adaptation network layer in the image fusion model, identifying the style of the initial facial feature information and the target ID photo template, and in the case that the style of the initial facial feature information does not match the style of the target ID photo template, adjusting the style of the initial facial feature information according to the style of the target ID photo template to obtain target facial feature information; inputting the target facial feature information and the target ID photo template into the feature merging network layer of the image fusion model, identifying the position information of the facial features in the target ID photo template, and according to the position information of the facial features in the target ID photo, fusing the facial feature information in the target ID photo template with the target facial feature information to obtain the first intermediate ID photo. The type of the fusion model is not limited in this embodiment. For example, the fusion model can be an Inswapper model, which is a deep learning model for face replacement function, and it can realize replacing the face of one person with the face of another person in an image or video. The said replacement is not a direct replacement of the face image, but a replacement of the facial features.

[0056] The above-mentioned first intermediate ID photo is obtained by preliminarily fusing the initial facial feature information with the target ID photo template, and the clarity of the facial features and the similarity with the face image of the user being photographed are low.

[0057] In order to improve the clarity of the facial features of the first intermediate ID photo, the first intermediate ID photo can be input into an image repair model to repair the face region in the first intermediate ID photo to obtain a second intermediate ID photo.

[0058] In some embodiments, a variational autoencoder (VAE) is also introduced, which consists of two parts, an encoder and a decoder. The encoder maps the input data to the distribution parameters (usually mean and variance) of a latent space, and the decoder generates data from the distribution of the latent space.

[0059] Based on this, in some embodiments, the image inpainting model includes an image defect detection network layer, an encoder, a repair network layer, and a decoder. Based on this, the face region in the first intermediate ID photo is repaired to obtain the second intermediate ID photo by inputting the first intermediate ID photo into the image inpainting model, which includes: inputting the first intermediate ID photo into the image defect detection network layer of the image inpainting model to detect the defect region of the first intermediate ID photo, and generating a mask image of the defect region under the condition that the defect region is detected. The mask image is a binary image used to identify the defect region in the image; further, inputting the face image (the originally obtained face image, and it is considered that the face features contained in the originally face image are complete and accurate), the first intermediate ID photo and the mask image into the encoder, the encoder uses a multi-layer convolutional neural network layer to extract the incomplete features contained in the local image corresponding to the mask image in the first intermediate ID photo, and the encoder uses a multi-layer convolutional neural network layer to extract the complete features contained in the local image corresponding to the mask image in the originally face image; further, inputting the incomplete features, the complete features, the first intermediate ID photo and the mask image into the repair network layer of the image inpainting model, repairing the incomplete features of the local image corresponding to the mask image in the first intermediate ID photo according to the complete features to obtain the repaired features; inputting the repaired features into the decoder to reconstruct the local face image corresponding to the defect region; fusing the reconstructed local face image with the first intermediate ID photo to obtain the second intermediate ID photo.

[0060] It should be noted that if the face features of the originally obtained face image are not clear and complete, the current clearest and most complete face features need to be reacquired for face image repair to obtain the second intermediate ID photo.

[0061] The image defect detection network layer can be regarded as a Code Attention module, which can use attention mechanisms to enhance the model's ability to capture information about the defect area. Specifically, to enable the model to perceive the relative positional relationships between different features, relative position encoding is introduced, which helps the model identify features around the defect area and better understand its structure. The conditional attention mechanism can adjust the attention distribution based on specific conditions such as the location and size of the defect area, allowing the model to focus more specifically on features related to the defect area. The multi-head self-attention mechanism can process information from different positions in parallel, allowing the model to capture information from different representation subspaces simultaneously, which helps better understand the context of the defect area. In addition, the Code Attention module can also establish connections between the encoder and the decoder, using features at different resolutions to provide more comprehensive context information, which helps the model capture details of the defect area at different scales.

[0062] The features provided by the decoder are multi-scale feature representations, and the repaired features are also multi-scale feature representations. When generating the second intermediate ID photo using the decoder, the decoder uses the multi-scale feature representations and the feature sequence generated by the Code Attention module to use the multi-head self-attention mechanism to process information from different representation subspaces in parallel, which helps capture global context information. The cross-attention layer allows the decoder's hidden state to interact with the encoder's output, combining the features extracted by the encoder with the code sequence. Layer normalization and residual connections in the decoder help stabilize the training process and prevent gradient vanishing or explosion problems. The spatial resolution of the feature map is gradually increased through the upsampling layers in the network structure, gradually restoring the details of the image. In addition, convolutional layers are used to further refine the feature map to generate a feature representation with the same size as the original image. Further, feature fusion techniques are used to fuse features at different levels to enhance the details and quality of the reconstructed image. Finally, a convolutional layer is used to convert the feature map to the final pixel values to obtain the second intermediate ID photo, which usually uses a tanh activation function to limit the range of pixel values to [-1, 1].

[0063] Further, to improve the clarity of the ID photo and the similarity to the user's face image taken, a text-to-image model can be introduced to further refine the second intermediate ID photo based on the text-to-image model to obtain the target ID photo.

[0064] The text-to-image model is used to output an image that meets the description of the text information based on the input image and text information after a certain number of iterations. The present embodiment does not limit the type of text-to-image model. For example, the text-to-image model can be an SD model. In addition, when selecting the SD model, a model that has generalization after pre-training, is adapted to the race to which the user belongs, and has high image clarity of the generated image can be selected, such as the Stable Diffusion XL (SDXL) model. The SDXL model is a new version of the SD model series, which has been significantly improved and optimized in image generation quality.

[0065] The text information refers to at least one prompt word corresponding to the target ID photo template, the gender of the user, and the actual needs of the user. The at least one prompt word includes at least a positive prompt word and a negative prompt word. The positive prompt word is a text description or keyword used to guide the model to generate a specific type or style and high-quality image. The negative prompt word is a text description or keyword used to guide the model to avoid generating certain specific features or elements. The negative prompt word is opposite to the positive prompt word, and both of them jointly act on the generation process of the model to achieve more accurate control of facial features. In addition, the present embodiment does not limit the way of obtaining the text information. For example, the text information is stored in association with each ID photo template, that is, each ID photo template is associated with the positive prompt word and the negative prompt word related to the ID photo template.

[0066] In some embodiments, in order to enhance the function and flexibility of the text-to-image model, a plug-in model can be integrated into the text-to-image model. The present embodiment does not specifically limit the type of plug-in model. For example, the plug-in model can be an InstantID model. The InstantID model includes a facial feature constraint model and a facial feature extraction model. The facial feature constraint model can be a Controlnet model, and the facial feature extraction model can be an Ipadapter model. The facial feature constraint model and the facial feature extraction model coexist, that is, they respectively perform image processing through decoupling cross-attention mechanisms without affecting each other.

[0067] Based on the above, further at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template can be input into the text-to-image model, and the second intermediate ID photo is finely processed according to the at least one prompt word to obtain the target ID photo. Optionally, at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template is input into the text-to-image model, and the second intermediate ID photo is finely processed according to the at least one prompt word to obtain the target ID photo, including: obtaining the first relative position information of the five features contained in the target face image of the user; using the five feature constraint model to constrain the position of the five features of the second intermediate ID photo to obtain the second relative position information of the constrained five features; and inputting the second intermediate ID photo into the five feature extraction model to obtain the five feature information of the second intermediate ID photo; determining the resolution of the target ID photo, creating a latent space of the target ID photo according to the resolution, and the vector dimension of the latent space is adapted to the feature space dimension of the target ID photo; using the encoder to map the second relative position information of the five features and the five feature information of the second intermediate ID photo to the latent space to obtain the latent space feature of the five features of the second intermediate ID photo in the latent space; according to at least one prompt word, using the sampler to collect five feature information adapted to at least one prompt word from the five feature information in the latent space; using the decoder to decode the five feature vector to obtain the target ID photo.

[0068] The above embodiments are mainly for fine processing of five feature information, but the skin color and texture of the target ID photo are not clear, and the similarity of the skin color and texture with the face image of the user is poor.

[0069] In order to solve the technical problem that the skin color and texture of the target ID photo are not clear and the similarity of the skin color and texture with the face image of the user is poor, in this embodiment, the target ID photo is input into the skin color enhancement magnification model, the target ID photo is magnified according to the magnification multiple of the skin color enhancement magnification model, and the skin color pixels of the magnified target ID photo are enhanced to obtain a first intermediate optimized ID photo. Obtain the first redraw area of the first intermediate optimized ID photo; input the first redraw area into the five feature extraction model to obtain the five feature information contained in the first redraw area; use the encoder to map the five feature information contained in the first redraw area to the latent space to obtain the latent space feature of the five feature information contained in the first redraw area in the latent space; according to at least one prompt word, using the sampler to collect five feature information adapted to at least one prompt word from the latent space feature corresponding to the five feature information contained in the first redraw area in the latent space; using the decoder to decode the five feature information adapted to at least one prompt word collected from the latent space feature corresponding to the five feature information contained in the first redraw area in the latent space to obtain the redrawn second intermediate optimized ID photo.

[0070] Optionally, the target ID photo is input into a skin color enhancement magnification model, the target ID photo is magnified according to a magnification multiple of the skin color enhancement magnification model, and the skin color pixels of the magnified target ID photo are enhanced to obtain a first intermediate optimized ID photo, including: obtaining image information of a skin color region of a face in the target ID photo, the image information including first color vectors of each pixel point in the skin color region, the first color vectors including a first red component color difference value and a first blue component color difference value; calculating a first skin color center region in the skin color region according to the first red component color difference value and the first blue component color difference value; obtaining image information of a skin color region of a face in the target ID photo magnified by the skin color enhancement magnification model, the image information including second color vectors of each pixel point in the skin color region, the second color vectors including a second red component color difference value and a second blue component color difference value; calculating a second skin color center region in the skin color region according to the second red component color difference value and the second blue component color difference value contained in the second color component; calculating an offset amount of the first color vectors and the second color vectors of each pixel point in the two skin color center regions according to the first red component color difference value, the first blue component color difference value, the first skin color center region, and the second red component color difference value, the second blue component color difference value, and the second skin color center region; and performing skin color enhancement processing on the image according to the offset amount. Thus, the selection of the skin color center region in the dynamic skin color model is set, so that the selection of the skin color center region is sensitive to the color difference of the skin color, and the real skin color in the current scene is truly reflected, so that the processing method has good skin color enhancement effect and wide applicability.

[0071] To improve the clarity of the first intermediate optimized ID photo, a second redrawing region of a second intermediate optimized ID photo can also be obtained; the second redrawing region is input into a facial feature extraction model to obtain facial feature information contained in the second redrawing region; the encoder is used to map the facial feature information contained in the second redrawing region to a latent space to obtain latent space features of the facial feature information contained in the second redrawing region in the latent space; the sampler is used to collect facial features adapted to the at least one prompt word from the latent space features corresponding to the facial feature information contained in the second redrawing region in the latent space according to the at least one prompt word; and the decoder is used to decode the facial features adapted to the at least one prompt word collected from the latent space features corresponding to the facial feature information contained in the second redrawing region in the latent space to obtain a redrawing optimized ID photo.

[0072] In addition, it should be noted that the sampler can also be introduced in each of the above embodiments. In image processing, the sampler can select or extract a specific part of the image or image set. When processing a face image, the sampler of DPM++ 2M Karras can be selected. The sampler has multiple functions, such as data sampling: in machine learning, the sampler is used to select a part of the data set for training or verification to improve the efficiency and performance of the model algorithm; feature point sampling: in image feature extraction, the sampler can be used to select key points or feature points, which are very important in image recognition and matching; image region selection: in image segmentation or target detection tasks, the sampler can be used to select the region of interest for more detailed processing; image fusion: in image synthesis or fusion, the sampler can be used to select the content in different images and merge them into a new image; image super-resolution: in image super-resolution technology, the sampler is used to extract feature information from low-resolution images to generate higher-resolution images; image denoising: the sampler can select key information in the image during the denoising process to reduce the impact of noise on image quality; image style conversion: in style conversion tasks, the sampler can be used to select and apply specific style features to the target image; image generation: when using a generative model, the sampler can be used to sample from the latent space to generate new images.

[0073] So far, the certificate photo generation method is executed. As shown in Figure 1b The contrast diagram of the initial acquired face image and the finally obtained optimized certificate photo is shown (the left side is the initial acquired face image, and the right side is the optimized certificate photo). It can be seen that the technical scheme provided by each of the above embodiments of the present application can generate a clear certificate photo of a user one key given a face image of the user with low clarity and a tilted face. Before image processing, a clear target certificate photo can be obtained without collecting multiple face images of the user, and the overall similarity between the target certificate photo and the original photo can be improved.

[0074] Figure 2 A flowchart of an image processing method provided by an exemplary embodiment of the present application is shown. As shown in Figure 2 , the method comprises:

[0075] 201, acquiring a face image of a target object, the face image containing initial facial feature information of the target object;

[0076] 202, identifying the gender attribute of the target object according to the initial facial feature information contained in the face image;

[0077] 203, selecting a target certificate photo template according to the gender attribute of the target object and a plurality of certificate photo templates corresponding to the gender attribute of the target object;

[0078] 204、fuse the initial facial feature information and the target ID photo template into the image fusion model, and fuse the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo;

[0079] 205、input the first intermediate ID photo into the image inpainting model, and inpaint the face region in the first intermediate ID photo to obtain a second intermediate ID photo;

[0080] 206、input at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into the text-to-image model, and perform fine processing on the second intermediate ID photo according to the at least one prompt word to obtain the target ID photo.

[0081] The detailed implementation and beneficial effects of each step provided in the embodiments of the present application have been described in detail in the foregoing embodiments, and will not be described in detail here.

[0082] Figure 3 A structural schematic diagram of an ID photo generation apparatus provided in an exemplary embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the apparatus includes: Figure 3

[0083] The acquisition module 31 is configured to acquire a face image of a user, and the face image contains initial facial feature information of the user.

[0084] The recognition module 32 is configured to recognize a gender attribute of the user according to the initial facial feature information contained in the face image.

[0085] The selection module 33 is configured to select a target ID photo template according to the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user.

[0086] The fusion module 34 is configured to fuse the initial facial feature information and the target ID photo template into the image fusion model, and fuse the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo.

[0087] The inpainting module 35 is configured to input the first intermediate ID photo into the image inpainting model, and inpaint the face region in the first intermediate ID photo to obtain a second intermediate ID photo.

[0088] The optimization module 36 is configured to input at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into the text-to-image model, and perform fine processing on the second intermediate ID photo according to the at least one prompt word to obtain the target ID photo.

[0089] ​In an optional embodiment, when the recognition module 32 identifies the gender attribute of the user according to the initial facial feature information contained in the facial image, it is specifically configured to: input the target facial image into the feature extraction network layer in the face recognition model to extract the facial feature, so as to obtain the facial feature information; input the facial feature information into the gender classification network layer of the face recognition model to classify the gender, so as to obtain the gender attribute of the user.

[0090] In another optional embodiment, when the recognition module 32 identifies the gender attribute of the user according to the facial feature information contained in the facial image, it is specifically configured to: input the facial image into the semantic segmentation model to perform semantic segmentation, so as to obtain a binary image containing the facial feature region; obtain a local image of the facial feature region according to the target facial image and the binary image, the local image being part of the target facial image; input the local image into the feature extraction network layer in the face recognition model to extract the facial feature, so as to obtain the facial feature information of the local image; input the facial feature information of the local image into the gender classification network layer in the face recognition model to classify the gender, so as to obtain the gender attribute of the user.

[0091] In some embodiments, when the selection module 33 selects the target ID photo template according to the gender attribute of the user and the plurality of ID photo templates corresponding to the gender attribute of the user, it is specifically configured to: select the plurality of ID photo templates corresponding to the gender attribute of the user according to the gender attribute of the user, and send the plurality of ID photo templates to the terminal device of the user, so that the terminal device displays the plurality of ID photo templates to the user; and receive a selection request sent by the terminal device in response to a selection operation of the user, and determine any ID photo template selected by the user as the target ID photo template according to the selection request. Alternatively, obtain the template requirement information of the user; determine the plurality of ID photo templates corresponding to the gender attribute of the user according to the gender attribute of the user; and select an ID photo template that matches the template requirement information of the user from the plurality of ID photo templates as the target ID photo template.

[0092] In some embodiments, the image fusion model comprises a style adaptation network layer and a feature merging network layer. When the fusion module 34 inputs the initial five-feature information and the target ID photo template into the image fusion model to fuse the initial five-feature information into the target ID photo template to obtain the first intermediate ID photo, specifically: input the initial five-feature information into the style adaptation network layer of the image fusion model, identify the styles of the initial five-feature information and the target ID photo template, and in the case that the styles of the initial five-feature information and the target ID photo template do not match, adjust the style of the initial five-feature information according to the style of the target ID photo template to obtain target five-feature information; input the target five-feature information and the target ID photo template into the feature merging network layer of the image fusion model, identify the position information of the five features in the target ID photo template, and according to the position information of the five features, fuse the five-feature information in the target ID photo template with the target five-feature information, i.e., correct the five-feature information in the target ID photo template to the target five-feature information, to obtain the first intermediate ID photo.

[0093] In some embodiments, when the repair module 35 inputs the first intermediate ID photo into the image repair model to repair the face region in the first intermediate ID photo to obtain the second intermediate ID photo, specifically: input the first intermediate ID photo into the image defect detection network layer of the image repair model to detect the defect region of the first intermediate ID photo, and in the case that the defect region is detected, generate a mask image of the defect region; input the face image of the user, the first intermediate ID photo and the mask image into an encoder, and use a multi-layer convolutional neural network layer to extract the incomplete face features contained in the local image corresponding to the mask image in the first intermediate ID photo and the complete face features contained in the local image corresponding to the face image; input the incomplete face features, the complete face features, the first intermediate ID photo and the mask image into the repair network layer of the image repair model, repair the incomplete face features of the local image corresponding to the mask image in the first intermediate ID photo according to the complete face features, to obtain repaired face features; input the repaired face features into a decoder, and reconstruct the local face image corresponding to the defect region by using the decoder; fuse the reconstructed local face image with the first intermediate ID photo to obtain the second intermediate ID photo.

[0094] In the embodiment, the text-to-image model is integrated with a plug-in model, and the plug-in model includes a facial feature constraint model and a facial feature extraction model. Based on this, when the optimization module 36 inputs at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into the text-to-image model, and performs fine processing on the second intermediate ID photo according to the at least one prompt word to obtain a target ID photo, it is specifically used for: obtaining first relative position information of facial features contained in the target facial image of the user; using the facial feature constraint model to constrain the positions of the facial features of the second intermediate ID photo to obtain second relative position information of the facial features after constraint; and inputting the second intermediate ID photo into the facial feature extraction model to obtain facial feature information of the second intermediate ID photo; determining a resolution of the target ID photo, creating a latent space of the target ID photo according to the resolution, and the vector dimension of the latent space is adapted to the feature space dimension of the target ID photo; using an encoder to map the second relative position information of the facial features and the facial feature information of the second intermediate ID photo to the latent space to obtain latent space features of the facial features of the second intermediate ID photo in the latent space; using a sampler to collect facial feature information adapted to the at least one prompt word from the latent space features of the facial features according to the at least one prompt word; and using a decoder to decode the facial feature vector to obtain the target ID photo.

[0095] Further, the optimization module 36 is also used for inputting the target ID photo into a skin color enhancement magnification model, magnifying the target ID photo according to a magnification multiple of the skin color enhancement magnification model and enhancing skin color pixels of the magnified target ID photo to obtain a first intermediate optimized ID photo; obtaining a first redraw area of the first intermediate optimized ID photo; inputting the first redraw area into the facial feature extraction model to obtain facial feature information contained in the first redraw area; using an encoder to map the facial feature information contained in the first redraw area to the latent space to obtain latent space features of the facial feature information contained in the first redraw area in the latent space; using a sampler to collect facial feature information adapted to the at least one prompt word from the latent space features corresponding to the facial feature information contained in the first redraw area according to the at least one prompt word; and using a decoder to decode the facial feature information collected from the latent space features corresponding to the facial feature information contained in the first redraw area according to the at least one prompt word to obtain a second intermediate optimized ID photo after redraw.

[0096] Further, the optimization module 36 is further configured to obtain a second redrawing area of the second intermediate optimized ID photo; input the second redrawing area into the facial feature extraction model to obtain facial feature information contained in the second redrawing area; map the facial feature information contained in the second redrawing area to the latent space by using the encoder to obtain latent space features of the facial feature information contained in the second redrawing area in the latent space; according to the at least one prompt word, collect facial features adapted to the at least one prompt word from the latent space features corresponding to the facial feature information contained in the second redrawing area by using the sampler; and decode the facial features adapted to the at least one prompt word collected from the latent space features corresponding to the facial feature information contained in the second redrawing area by using the decoder to obtain the redrawing optimized ID photo.

[0097] The detailed implementation and beneficial effects of each module provided in the embodiments of the present application have been described in detail in the foregoing embodiments, and will not be described in detail here.

[0098] Figure 4 The structure schematic diagram of the image processing device provided in the exemplary embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, it comprises: Figure 4

[0099] The acquisition module 41 is configured to acquire a facial image of a target object, and the facial image contains initial facial feature information of the target object.

[0100] The recognition module 42 is configured to recognize the gender attribute of the target object according to the initial facial feature information contained in the facial image.

[0101] The selection module 43 is configured to select a target ID photo template according to the gender attribute of the target object and a plurality of ID photo templates corresponding to the gender attribute of the target object.

[0102] The fusion module 44 is configured to input the initial facial feature information and the target ID photo template into an image fusion model, and fuse the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo.

[0103] The repair module 45 is configured to input the first intermediate ID photo into an image repair model, and repair the facial region in the first intermediate ID photo to obtain a second intermediate ID photo.

[0104] The optimization module 46 is configured to input the second intermediate ID photo and at least one prompt word corresponding to the target ID photo template into a text-to-image model, and perform fine processing on the second intermediate ID photo according to the at least one prompt word to obtain a target ID photo.

[0105] ​The detailed implementation and beneficial effects of each module provided in the embodiments of the present application have been described in the foregoing embodiments, and will not be described in detail here.

[0106] Figure 5 A structural schematic diagram of an electronic device is provided for an exemplary embodiment of the present application. As shown in the figure, it includes a memory 50a and a processor 50b; the memory 50a is used to store a computer program; the processor 50b is coupled with the memory 50a and is used to execute the computer program to implement the following steps: Figure 5

[0107] obtaining a face image of a user, the face image containing initial facial feature information of the user; identifying a gender attribute of the user according to the initial facial feature information contained in the face image; selecting a target ID photo template according to the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user; inputting the initial facial feature information and the target ID photo template into an image fusion model, fusing the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo; inputting the first intermediate ID photo into an image repair model, repairing a face region in the first intermediate ID photo to obtain a second intermediate ID photo; inputting at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into a text-to-image model, and performing fine processing on the second intermediate ID photo according to the at least one prompt word to obtain a target ID photo.

[0108] In an optional embodiment, when the processor 50b identifies the gender attribute of the user according to the initial facial feature information contained in the face image, it is specifically used for: inputting the target face image into a feature extraction network layer in the face recognition model to extract facial features to obtain facial feature information; inputting the facial feature information into a gender classification network layer of the face recognition model to perform gender classification to obtain the gender attribute of the user.

[0109] In another optional embodiment, when the processor 50b identifies the gender attribute of the user according to the facial feature information contained in the face image, it is specifically used for: inputting the face image into a semantic segmentation model to perform semantic segmentation to obtain a binary image containing facial feature regions; obtaining a local image of the facial feature regions according to the target face image and the binary image, the local image being part of the target face image; inputting the local image into the feature extraction network layer in the face recognition model to extract facial features to obtain facial feature information of the local image; inputting the facial feature information of the local image into the gender classification network layer in the face recognition model to perform gender classification to obtain the gender attribute of the user.

[0110] ​In some embodiments, when the processor 50b selects the target ID photo template according to the gender attribute of the user and the plurality of ID photo templates corresponding to the gender attribute of the user, specifically: according to the gender attribute of the user, selecting a plurality of ID photo templates corresponding to the gender attribute of the user, and sending the plurality of ID photo templates to the terminal device of the user, so that the terminal device displays the plurality of ID photo templates to the user; and receiving the selection request sent by the terminal device in response to the selection operation of the user, and determining any ID photo template selected by the user as the target ID photo template according to the selection request. Alternatively, obtaining the template requirement information of the user; according to the gender attribute of the user, determining a plurality of ID photo templates corresponding to the gender attribute of the user; selecting an ID photo template that matches the template requirement information of the user from the plurality of ID photo templates as the target ID photo template.

[0111] In some embodiments, the image fusion model includes a style adaptation network layer and a feature merging network layer. When the processor 50b inputs the facial feature information and the target ID photo template into the image fusion model to fuse the initial facial feature information into the target ID photo template to obtain the first intermediate ID photo, specifically: input the initial facial feature information into the style adaptation network layer in the image fusion model, identify the style of the initial facial feature information and the target ID photo template, and in the case that the style of the initial facial feature information does not match the style of the target ID photo template, adjust the style of the initial facial feature information according to the style of the target ID photo template to obtain target facial feature information; input the target facial feature information and the target ID photo template into the image fusion model, identify the position of the facial features in the target ID photo template, and according to the position information of the facial features, correspondingly correct the facial feature information in the target ID photo template to the target facial feature information to obtain the first intermediate ID photo.

[0112] In some embodiments, when the processor 50b inputs the first intermediate ID photo into the image inpainting model to inpaint the face region of the first intermediate ID photo to obtain a second intermediate ID photo, specifically, the processor 50b is configured to: detect the first intermediate ID photo in the image defect detection network layer of the image inpainting model to detect a defect region, and generate a mask image of the defect region in a case where the defect region is detected; input the face image of the user, the first intermediate ID photo, and the mask image into an encoder, and use a multi-layer convolutional neural network layer to extract a residual face feature contained in a local image corresponding to the mask image in the first intermediate ID photo and a complete face feature contained in a local image corresponding to the face image; input the residual face feature, the complete face feature, the first intermediate ID photo, and the mask image into a repair network layer of the image inpainting model, and repair the residual face feature of the local image corresponding to the mask image in the first intermediate ID photo according to the complete face feature to obtain a repaired face feature; input the repaired face feature into a decoder, and use the decoder to reconstruct a local face image corresponding to the defect region; and fuse the reconstructed local face image with the first intermediate ID photo to obtain the second intermediate ID photo.

[0113] In the present embodiment, the text-to-image model is integrated with a plug-in model, and the plug-in model includes a facial feature constraint model and a facial feature extraction model. Based on this, after obtaining the second intermediate ID photo, the processor 50b is configured to input the second intermediate ID photo and at least one prompt word corresponding to the target ID photo template into the text-to-image model, and perform fine processing on the second intermediate ID photo according to the at least one prompt word to obtain a target ID photo, specifically: obtaining first relative position information of facial features contained in a target face image of the user; using the facial feature constraint model to constrain the positions of the facial features of the second intermediate ID photo to obtain second relative position information of the facial features after the constraint; and inputting the second intermediate ID photo into the facial feature extraction model to obtain facial feature information of the second intermediate ID photo; determining a resolution of the target ID photo, creating a latent space of the target ID photo according to the resolution, and the vector dimension of the latent space is adapted to the feature space dimension of the target ID photo; using an encoder to map the second relative position information of the facial features and the facial feature information of the second intermediate ID photo to the latent space to obtain latent space features of the facial features of the second intermediate ID photo in the latent space; using a sampler to collect facial feature vectors adapted to the at least one prompt word from the latent space features of the facial features according to the at least one prompt word; and using a decoder to decode the facial feature vectors to obtain the target ID photo.

[0114] Further, the processor 50b is further configured to input the target ID photo into a skin color enhancement magnification model, magnify the target ID photo according to a magnification multiple of the skin color enhancement magnification model, and enhance skin color pixels of the magnified target ID photo to obtain a first intermediate optimized ID photo; obtain a first redrawn area of the first intermediate optimized ID photo; input the first redrawn area into the facial feature extraction model to obtain facial feature information contained in the first redrawn area; map the facial feature information contained in the first redrawn area to the latent space by using an encoder to obtain latent space features of the facial feature information contained in the first redrawn area in the latent space; according to the at least one prompt word, collect facial features adapted to the at least one prompt word from the latent space features corresponding to the facial feature information contained in the first redrawn area in the latent space by using a sampler; and decode the facial features adapted to the at least one prompt word collected from the latent space features corresponding to the facial feature information contained in the first redrawn area in the latent space by using a decoder to obtain a second intermediate optimized ID photo after redrawing.

[0115] Further, the processor 50b is further configured to obtain a second redrawn area of the second intermediate optimized ID photo; input the second redrawn area into the facial feature extraction model to obtain facial feature information contained in the second redrawn area; map the facial feature information contained in the second redrawn area to the latent space by using an encoder to obtain latent space features of the facial feature information contained in the second redrawn area in the latent space; according to the at least one prompt word, collect facial features adapted to the at least one prompt word from the latent space features corresponding to the facial feature information contained in the second redrawn area in the latent space by using a sampler; and decode the facial features adapted to the at least one prompt word collected from the latent space features corresponding to the facial feature information contained in the second redrawn area in the latent space by using a decoder to obtain an optimized ID photo after redrawing.

[0116] In addition, based on the same structure of the electronic device, the processor 50b is further configured to perform the following steps:

[0117] The face image of the target object is acquired, and the face image contains initial facial feature information of the target object; a gender attribute of the target object is recognized according to the initial facial feature information contained in the face image; a target photo template is selected according to the gender attribute of the target object and a plurality of photo templates corresponding to the gender attribute of the target object; the initial facial feature information and the target photo template are input into an image fusion model, the initial facial feature information is fused into the target photo template to obtain a first intermediate photo; the first intermediate photo is input into an image repair model, and a face region in the first intermediate photo is repaired to obtain a second intermediate photo; at least one prompt word corresponding to the second intermediate photo and the target photo template is input into a text-to-image model, and the second intermediate photo is finely processed according to the at least one prompt word to obtain a target photo.

[0118] Further, as shown in Figure 5 The electronic device further includes a communication component 50c, a display 50d, a power supply component 50e, an audio component 50f, and other components. Figure 5 Some components are only schematically shown, and it does not mean that the electronic device only includes Figure 5 the components shown.

[0119] The detailed implementation and beneficial effects of the electronic device provided in the embodiments of the present application have been described in the foregoing embodiments, and will not be described in detail here.

[0120] The exemplary embodiments of the present application also provide a computer readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the processor is caused to implement the steps in the above method embodiments.

[0121] The exemplary embodiments of the present application also provide a computer program product, which contains computer programs / instructions, and when the computer programs / instructions are executed by a processor, the processor is caused to be able to implement the steps in the above method embodiments.

[0122] The above-described memory can be implemented by any type of volatile or nonvolatile memory devices or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0123] The above-described communication component is configured to facilitate communication between the device in which the communication component is located and other devices in a wired or wireless manner. The device in which the communication component is located can access a wireless network based on a communication standard, such as a WiFi, 2G, 3G, 4G / LTE, 5G, or the like mobile communication network, or a combination thereof. In an example embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast managing system via a broadcast channel. In an example embodiment, the communication component further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Blue Tooth (BT) technology, and other technologies.

[0124] The above-described display includes a screen, which can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touch or a slide action, but also detect a duration and a pressure related to a touch or a slide operation.

[0125] The power supply component provides power to various components of the device in which the power supply component is located. The power supply component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0126] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0127] The present application is described in reference to the flowchart illustrations and / or block diagrams according to the embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0128] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams block or blocks.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer implemented process such that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0130] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0131] Memory can include non-persistent memory in computer-readable media, Random Access Memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory, etc. Memory is an example of computer-readable media.

[0132] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital video disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0133] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article or apparatus that includes the element.

[0134] The above only describes the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method of generating an ID photo, characterized by, The method comprises the following steps: obtaining a face image of a user, the face image containing initial facial feature information of the user; identifying a gender attribute of the user according to the initial facial feature information contained in the face image; selecting a target ID photo template according to the gender attribute of the user, template requirement information of the ID photo template, and a plurality of ID photo templates corresponding to the gender attribute of the user; inputting the initial facial feature information and the target ID photo template into an image fusion model, fusing the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo; inputting the first intermediate ID photo into an image repair model, repairing a face region in the first intermediate ID photo to obtain a second intermediate ID photo; inputting the second intermediate ID photo and at least one prompt word corresponding to the target ID photo template into a text-to-image model, and performing fine processing on the second intermediate ID photo according to the at least one prompt word to obtain a target ID photo; wherein the image fusion model comprises a style adaptation network layer and a feature merging network layer; inputting the initial facial feature information and the target ID photo template into the image fusion model, fusing the initial facial feature information into the target ID photo template to obtain the first intermediate ID photo comprises: inputting the initial facial feature information into the style adaptation network layer in the image fusion model, identifying the style of the initial facial feature information and the target ID photo template, and in the case that the style of the initial facial feature information does not match the style of the target ID photo template, adjusting the style of the initial facial feature information according to the style of the target ID photo template to obtain target facial feature information; inputting the target facial feature information and the target ID photo template into the feature merging network layer of the image fusion model, identifying the positions of the facial features in the target ID photo template, and according to the position information of the facial features, correcting the facial feature information in the target ID photo template to the target facial feature information to obtain the first intermediate ID photo.

2. The method of claim 1, wherein, According to the initial facial feature information contained in the face image, identifying the gender attribute of the user comprises: inputting the face image into a feature extraction network layer in a face recognition model to extract facial feature information to obtain the facial feature information; inputting the facial feature information into a gender classification network layer of the face recognition model to classify the gender to obtain the gender attribute of the user.

3. The method of claim 1, wherein, According to the facial feature information contained in the face image, identifying the gender attribute of the user comprises: inputting the face image into a semantic segmentation model to perform semantic segmentation to obtain a binary image containing a facial feature region; obtaining a local image of the facial feature region according to the target face image and the binary image, the local image being a partial image in the target face image; inputting the local image into a feature extraction network layer in a face recognition model to extract facial feature information of the local image to obtain the facial feature information of the local image. The facial feature information of the local image is input into a gender classification network layer in the face recognition model for gender classification to obtain a gender attribute of the user.

4. The method of claim 1, wherein, According to the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user, a target ID photo template is selected, including: According to the gender attribute of the user, a plurality of ID photo templates corresponding to the gender attribute of the user are selected, and the plurality of ID photo templates are sent to a terminal device of the user, so that the terminal device displays the plurality of ID photo templates to the user; and a selection request sent by the terminal device in response to a selection operation of the user is received, and any ID photo template selected by the user is determined as the target ID photo template according to the selection request; Or, The template requirement information of the user is obtained; according to the gender attribute of the user, a plurality of ID photo templates corresponding to the gender attribute of the user are determined; and an ID photo template adapted to the template requirement information of the user is selected from the plurality of ID photo templates as the target ID photo template.

5. The method of claim 1, wherein, The first intermediate ID photo input image is input into an image inpainting model, and the face region of the first intermediate ID photo is inpainted to obtain a second intermediate ID photo, including: The first intermediate ID photo is input into an image defect detection network layer of the image inpainting model, and the defect region of the first intermediate ID photo is detected, and a mask image of the defect region is generated in the case of detecting the defect region; The first intermediate ID photo and the mask image are input into a repair network layer of the image inpainting model, and the mask image is repaired according to the first intermediate ID photo to obtain a preliminarily repaired first intermediate ID photo; The face region of the preliminarily repaired first intermediate ID photo is identified by inputting the preliminarily repaired first intermediate ID photo into a face detection network layer of the image inpainting model to obtain a face image of the preliminarily repaired first intermediate ID photo; The face image of the preliminarily repaired first intermediate ID photo is input into a face reconstruction network layer of the image inpainting model to obtain a reconstructed face image; The reconstructed face image is pasted back to the preliminarily repaired first intermediate ID photo, and the preliminarily repaired first intermediate ID photo with the reconstructed face image pasted back is subjected to super-resolution processing by an image super-resolution network layer of a high-definition inpainting model to obtain a second intermediate ID photo.

6. The method of claim 1, wherein, The text-to-image model integrates a plug-in model, and the plug-in model includes a facial feature constraint model; After obtaining the second intermediate ID photo, the method further includes obtaining first relative position information of the facial features contained in the target face image of the user; The position of the facial features of the second intermediate ID photo is constrained by using the facial feature constraint model to obtain second relative position information of the facial features after the constraint; and the second intermediate ID photo is input into the facial feature extraction model to obtain facial feature information of the second intermediate ID photo; determining a resolution of a target ID photo, creating a latent space of the target ID photo according to the resolution, a vector dimension of the latent space being adapted to a feature space dimension of the target ID photo; mapping, by using an encoder, the second relative position information of the five features and the five feature information of the second intermediate ID photo to the latent space to obtain latent space features of the five features of the second intermediate ID photo in the latent space; collecting, by using a sampler, five feature information adapted to the at least one prompt word from the latent space features of the five features according to the at least one prompt word; decoding, by using a decoder, the five feature vectors to obtain the target ID photo.

7. The method of claim 6, wherein, Further comprising: inputting the target ID photo into a skin color enhancement magnification model, magnifying the target ID photo according to a magnification multiple of the skin color enhancement magnification model, and enhancing skin color pixels of the magnified target ID photo to obtain a first intermediate optimized ID photo; obtaining a first redrawing area of the first intermediate optimized ID photo; inputting the first redrawing area into the five feature extraction model to obtain five feature information contained in the first redrawing area: mapping, by using an encoder, the five feature information contained in the first redrawing area to the latent space to obtain latent space features of the five feature information contained in the first redrawing area in the latent space; collecting, by using a sampler, five feature information adapted to the at least one prompt word from the latent space features of the five feature information contained in the first redrawing area in the latent space according to the at least one prompt word; decoding, by using a decoder, the five feature information adapted to the at least one prompt word collected from the latent space features of the five feature information contained in the first redrawing area in the latent space to obtain a second intermediate optimized ID photo after redrawing.

8. The method of claim 7, wherein, Further comprising: obtaining a second redrawing area of the second intermediate optimized ID photo; inputting the second redrawing area into the five feature extraction model to obtain five feature information contained in the second redrawing area: mapping, by using an encoder, the five feature information contained in the second redrawing area to the latent space to obtain latent space features of the five feature information contained in the second redrawing area in the latent space; collecting, by using a sampler, five feature information adapted to the at least one prompt word from the latent space features of the five feature information contained in the second redrawing area in the latent space according to the at least one prompt word; decoding, by using a decoder, the five feature information adapted to the at least one prompt word collected from the latent space features of the five feature information contained in the second redrawing area in the latent space to obtain an optimized ID photo after redrawing.

9. An image processing method characterized by, comprising: obtaining a face image of a target object, the face image containing initial five feature information of the target object; identifying a gender attribute of the target object according to the initial five feature information contained in the face image; selecting a target ID photo template according to the gender attribute of the target object, template requirement information of an ID photo template, and a plurality of ID photo templates corresponding to the gender attribute of the target object; Fusing the initial five facial feature information and the target ID photo template into an image fusion model, fusing the initial five facial feature information into the target ID photo template to obtain a first intermediate ID photo; Inputting the first intermediate ID photo into an image repairing model, repairing the face region in the first intermediate ID photo to obtain a second intermediate ID photo; Inputting the second intermediate ID photo and at least one prompt word corresponding to the target ID photo template into a text-to-image model, and performing fine processing on the second intermediate ID photo according to the at least one prompt word to obtain a target ID photo; The image fusion model includes a style adaptation network layer and a feature merging network layer; fusing the initial five facial feature information and the target ID photo template into an image fusion model, fusing the initial five facial feature information into the target ID photo template to obtain a first intermediate ID photo includes: Inputting the initial five facial feature information into the style adaptation network layer in the image fusion model, identifying the style of the initial five facial feature information and the target ID photo template, and in the case that the style of the initial five facial feature information does not match the style of the target ID photo template, adjusting the style of the initial five facial feature information according to the style of the target ID photo template to obtain target five facial feature information; Inputting the target five facial feature information and the target ID photo template into the feature merging network layer of the image fusion model, identifying the positions of the five facial features in the target ID photo template, and according to the position information of the five facial features, correcting the five facial feature information in the target ID photo template to the target five facial feature information to obtain a first intermediate ID photo.

10. An electronic device, comprising: comprising: a memory and a processor; the memory is configured to store a computer program; the processor is coupled to the memory and is configured to execute the computer program in the memory to implement the steps in the method of any one of claims 1-8 or claim 9.

11. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the processor can implement the steps in the method of any one of claims 1-8 or claim 9.

12. A computer program product, characterised in that, comprising computer programs / instructions, which, when executed by the processor, enable the processor to implement the steps in the method of any one of claims 1-8 or claim 9.

Images