Image processing and identification photo generation method and device, storage medium and program product

By identifying the user's gender attributes and integrating, repairing and refined processing techniques, high-definition ID photos are generated, which solves the problem of low clarity and similarity of ID photos in the existing technology, and simplifies the generation process.

CN119006636AActive Publication Date: 2024-11-22BEIJING CHENGSHI WANGLIN INFORMATION TECH CO LTD

Patent Information

Application Number
CN202411141128.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-22
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

When generating ID photos in the prior art, the clarity and similarity to the original photos are low, and multiple face images are required to be input, which is a cumbersome process.

Method used

By obtaining the user's face image, identifying gender attributes, selecting the corresponding ID photo template, using the image fusion model to fusion five-feature characteristics, the image repair model repairs the face area, and the literary and graphic model performs fine processing to generate high-definition ID photo .

Benefits of technology

High-definition ID photos can be generated without multiple face images, which improves the overall similarity between ID photos and the original photos and simplifies the generation process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006636A_ABST
    Figure CN119006636A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an image processing and identification photo generation method and device, a storage medium and a program product. In the embodiment of the invention, the gender attribute of the user can be identified according to the initial feature information of the five sense organs contained in the face image; the gender attribute of the user corresponds to a plurality of identification photo templates, and a target identification photo template can be selected from the plurality of identification photo templates; further, the initial five-sense-organ feature information can be fused into a target identification photo template by using an image fusion model, so that a first intermediate identification photo is obtained; furthermore, the image restoration model is utilized to restore the face region in the first intermediate identification photo to obtain the second intermediate identification photo, so that the face region of the identification photo is clearer. Afterwards, the second intermediate identification photo can be refined according to the at least one prompt word by using the text graph model to obtain a target identification photo, so that the definition of the target identification photo and the overall similarity of the target identification photo and the original photo are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing and ID photo generation method, device, storage medium and program product. Background Art

[0002] In daily life, study, and work, we often need to take various types of photos, such as ID photos required for work IDs, student IDs, driver's licenses, wedding photos, etc. For the same type of ID photos, there are many styles of photos and they have certain requirements on size, color, and appearance. Usually, professional photographers are needed to collect photos, but it is not convenient for professional photographers to collect photos.

[0003] With the continuous development of image processing technology, various photo applications have emerged. At present, most photo applications on the market generate photos based on generative algorithms. Specifically, multiple facial images of the user are input, and the multiple facial images are simply processed using the image processing model. The processed facial images are pasted onto the ID photo template to obtain the ID photo.

[0004] However, the ID photos generated in the above manner are of a single type, and the clarity of the ID photos and the overall similarity with the original photos are low. Summary of the invention

[0005] Multiple aspects of the present application provide an image processing and ID photo generation method, device, storage medium and program product to improve the clarity of the generated photo and the overall similarity with the original photo.

[0006] The embodiment of the present application provides a method for generating an ID photo, including: obtaining a face image of a user, the face image including initial facial feature information of the user; identifying the gender attribute of the user based on the initial facial feature information included in the face image; selecting a target ID photo template based on the gender attribute of the user and multiple ID photo templates corresponding to the gender attribute of the user; inputting the initial facial feature information and the target ID photo template into an image fusion model, and fusing the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo; inputting the first intermediate ID photo into an image restoration model, and repairing the face area in the first intermediate ID photo to obtain a second intermediate ID photo; inputting at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into a text map model, and performing fine processing on the second intermediate ID photo based on the at least one prompt word to obtain a target ID photo.

[0007] The embodiment of the present application also provides an image processing method, including: obtaining a facial image of a target object, the facial image including initial facial feature information of the target object; identifying the gender attribute of the target object based on the initial facial feature information included in the facial image; selecting a target photo template based on the gender attribute of the target object and multiple photo templates corresponding to the gender attribute of the target object; inputting the initial facial feature information and a target ID photo template into an image fusion model, and fusing the initial facial feature information into the target photo template to obtain a first intermediate photo; inputting the first intermediate ID photo into an image restoration model, and repairing the facial area in the first intermediate photo to obtain a second intermediate photo; inputting at least one prompt word corresponding to the second intermediate photo and the target photo template into a Vincent graph model, and performing fine processing on the second intermediate photo according to the at least one prompt word to obtain a target photo.

[0008] An embodiment of the present application also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program in the memory, so as to implement the steps in the ID photo generation method or image processing method provided in the embodiment of the present application.

[0009] The embodiments of the present application also provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement the steps in the ID photo generation method or image processing method provided in the embodiments of the present application.

[0010] An embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement the steps in the ID photo generation method or the image processing method.

[0011] The technical solution provided by the embodiment of the present application, on the one hand, for the generation of user ID photos, multiple image models can be used to perform multi-stage image processing according to the user's facial image, specifically, the user's gender attribute can be identified according to the initial facial features information contained in the facial image; the user's gender attribute corresponds to multiple ID photo templates, thereby selecting a target ID photo template from the multiple ID photo templates; further, the image fusion model can be used to fuse the initial facial features information into the target ID photo template, thereby obtaining a first intermediate ID photo; further, the image repair model can be used to repair the face area in the first intermediate ID photo to obtain a second intermediate ID photo, so that the face area of ​​the ID photo is clearer. Afterwards, when a clear second intermediate ID photo is obtained, the Wensheng graph model can be used to perform fine processing on the second intermediate ID photo according to at least one prompt word to obtain the target ID photo. Therefore, before image processing, it is not necessary to collect multiple facial images of the user to obtain a target ID photo with high clarity, and the overall similarity between the target ID photo and the original photo can also be improved.

[0012] On the other hand, for the generation of various types of photos of the target object, multiple image models can be used to perform multi-stage image processing based on the facial image of the target object. Specifically, the gender attribute of the target object can be identified based on the initial facial features information contained in the facial image; the gender attribute of the target object corresponds to multiple photo templates, and the target photo template is selected from the multiple photo templates; further, the image fusion model can be used to fuse the initial facial features information into the target photo template to obtain a first intermediate photo; further, the image restoration model can be used to restore the facial area in the first intermediate photo to obtain a second intermediate photo, so that the facial area of ​​the photo is clearer. Afterwards, when a clear second intermediate photo is obtained, the Wensheng graph model can be used to perform fine processing on the second intermediate photo according to at least one prompt word to obtain a target photo. Therefore, before image processing, a target photo with high clarity can be obtained without collecting multiple facial images of the target object, and the overall similarity between the target photo and the original photo can also be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0014] Figure 1a A flowchart of a method for generating a certificate photo provided by an exemplary embodiment of the present application;

[0015] Figure 1b A schematic diagram of the comparison between an original face image and an optimized ID photo provided in an exemplary embodiment of the present application;

[0016] Figure 2 A flowchart of an image processing method provided by an exemplary embodiment of the present application;

[0017] Figure 3 A schematic diagram of the structure of a device for generating a certificate photo provided by an exemplary embodiment of the present application;

[0018] Figure 4 A schematic diagram of the structure of an image processing device provided by an exemplary embodiment of the present application;

[0019] Figure 5 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0021] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation portals for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) are in compliance with relevant laws and standards.

[0022] At present, most of the photo-taking software on the market generates photos based on generative algorithms. Specifically, multiple original images of human faces are input, and the multiple original images of human faces are simply processed using image processing models to obtain the type of photos the user wants. However, the photos generated in this way have certain requirements for the input original images, such as the need to input multiple original images and the original images must be frontal photos, and the resolution and facial clarity cannot be too low, and the type of ID photos generated is single; in addition, the overall similarity between the ID photos generated in this way and the original images is not high.

[0023] In view of the above technical problems, in the embodiments of the present application, on the one hand, for the generation of user ID photos, multiple image models can be used to perform multi-stage image processing according to the user's face image, specifically, the user's gender attribute can be identified according to the initial facial feature information contained in the face image; the user's gender attribute corresponds to multiple ID photo templates, thereby selecting a target ID photo template from the multiple ID photo templates; further, the image fusion model can be used to fuse the initial facial feature information into the target ID photo template, thereby obtaining a first intermediate ID photo; further, the image repair model can be used to repair the face area in the first intermediate ID photo to obtain a second intermediate ID photo, so that the face area of ​​the ID photo is clearer. Afterwards, when a clear second intermediate ID photo is obtained, the Wensheng graph model can be used to perform fine processing on the second intermediate ID photo according to at least one prompt word to obtain the target ID photo. Therefore, before image processing, it is not necessary to collect multiple facial images of the user to obtain a target ID photo with high clarity, and the overall similarity between the target ID photo and the original photo can also be improved.

[0024] On the other hand, for the generation of various types of photos of the target object, multiple image models can be used to perform multi-stage image processing based on the facial image of the target object. Specifically, the gender attribute of the target object can be identified based on the initial facial features information contained in the facial image; the gender attribute of the target object corresponds to multiple photo templates, and the target photo template is selected from the multiple photo templates; further, the image fusion model can be used to fuse the initial facial features information into the target photo template to obtain a first intermediate photo; further, the image restoration model can be used to restore the facial area in the first intermediate photo to obtain a second intermediate photo, so that the facial area of ​​the photo is clearer. Afterwards, when a clear second intermediate photo is obtained, the Wensheng graph model can be used to perform fine processing on the second intermediate photo according to at least one prompt word to obtain a target photo. Therefore, before image processing, a target photo with high clarity can be obtained without collecting multiple facial images of the target object, and the overall similarity between the target photo and the original photo can also be improved.

[0025] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0026] Figure 1b The following is a flow chart of a method for generating a certificate photo provided by an exemplary embodiment of the present application. Figure 1b As shown, including:

[0027] 101. Obtain a facial image of the user, where the facial image includes initial facial feature information of the user;

[0028] 102. Identify the gender attribute of the user based on the initial facial features information contained in the facial image;

[0029] 103. Select a target ID photo template according to the gender attribute of the user and multiple ID photo templates corresponding to the gender attribute of the user;

[0030] 104. Inputting the initial facial feature information and the target ID photo template into the image fusion model, and fusing the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo;

[0031] 105. Input the first intermediate ID photo into the image restoration model, and perform restoration on the face region in the first intermediate ID photo to obtain a second intermediate ID photo;

[0032] 106. Input at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into the text graph model, and perform fine processing on the second intermediate ID photo according to the at least one prompt word to obtain the target ID photo.

[0033] In the embodiment of the present application, the user may be a person who needs to generate an ID photo for different positions. This embodiment does not limit the types of positions. For example, the position may be a chef position, a lawyer position, an administrative staff position or a domestic staff position, and so on. Different positions may have different requirements for ID photos. For a chef position, the ID photo may require a chef's hat; for a lawyer position, the ID photo may require a suit and tie; for an administrative staff position, the ID photo may require a bun or short hair; for a domestic staff position, the ID photo may require domestic clothing, and so on. This embodiment does not specifically limit this. Moreover, for the same position, different recruitment companies may have different requirements for ID photos. In addition, different positions or different recruitment companies may also have different requirements for the background color of the ID photo. For example, the background color may be white, blue, or red, and so on.

[0034] In an embodiment of the present application, multiple types of ID photo templates can be provided, and multiple templates of the same type of ID photos can also be provided for users to choose from, so as to increase the diversity of ID photos.

[0035] Furthermore, in order to enable users to accurately select ID photo templates, in this embodiment, users can also be divided according to gender attributes, and multiple ID photo templates are provided for users of different gender attributes, so that users can choose ID photo templates that meet their needs. Specifically, a user's facial image can be obtained, and the facial image contains the user's initial facial feature information; further, the user's gender attribute is identified based on the initial facial feature information contained in the facial image; then, the target ID photo is selected based on the user's gender attribute and multiple ID photo templates corresponding to the user's gender attribute. Among them, the facial features at least include the user's eyebrows, eyes, nose, mouth and ears. The initial facial feature information at least includes the feature information of the user's eyebrows, eyes, nose, mouth and ears contained in the facial image, and the feature information specifically refers to the size, position, color and shape of the above facial features.

[0036] In this embodiment, the method of obtaining the user's facial image is not limited. For example, when generating an ID photo, the camera of the terminal device installed with the camera application can be called to capture the user's facial image in real time; or, the user's facial image captured in advance can be selected from the album of the terminal device installed with the camera application; or, the user's facial image captured in advance can be obtained from other terminal devices or databases, etc. In addition, this embodiment does not limit the posture of the user when capturing the facial image, as long as the facial image contains the user's facial feature information. For example, it can be a facial image of a sitting posture, a facial image in a tilted posture, or a facial image in a lying posture, etc. In addition, this embodiment does not limit the number of user's facial images to be obtained, as long as the facial image contains the user's facial feature information. For example, only one user's facial image can be obtained, and an ID photo can be generated based on one facial image; or, multiple user's facial images can be obtained, and an ID photo can be generated based on multiple facial images, etc. In this embodiment, the focus is on obtaining one ID photo of the user's face, thereby reducing the number of ID photos used and reducing the workload of facial image shooting.

[0037] In an optional embodiment, the user's initial facial feature information can be directly extracted from the user's facial image, and the user's gender attribute can be identified based on the initial facial feature information. Specifically, the user's gender attribute is identified based on the initial facial feature information contained in the facial image, including: inputting the target facial image into the feature extraction network layer in the facial recognition model to extract the initial facial features to obtain the initial facial feature information; inputting the initial facial feature information into the gender classification network layer of the facial recognition model to perform gender classification to obtain the user's gender attribute. This embodiment does not limit the type of facial recognition model. For example, the facial recognition model can be an Insight Face model. InsightFace is a facial recognition system based on deep learning. It is efficient, accurate and scalable. It can use advanced neural network technology to identify and verify faces in various complex environments.

[0038] Optionally, the target face image is input into the feature extraction network layer in the face recognition model to extract facial features to obtain facial feature information, including: first extracting primary features of the image, such as edges and textures of facial features; gradually combining the primary features into more complex feature representations, which contain initial facial feature information; extracting features according to the feature level to be extracted to obtain facial feature information.

[0039] In another optional embodiment, the semantic segmentation model can also be used to perform semantic segmentation on the face image, and the user's gender attribute can be identified based on the initial facial feature information contained in the local image after semantic segmentation. The semantic segmentation model can be used to segment a more accurate local image containing the initial facial feature information, and the initial facial feature information can be extracted based on the accurate local image, which can reduce the amount of data processing on the one hand, and improve the accuracy of the extracted initial facial feature information on the other hand.

[0040] Among them, semantic segmentation refers to a technology in the field of computer vision and image processing, the purpose of which is to assign each pixel in the image to a specific category or object, so as to realize the recognition and segmentation of all objects in the image, such as the recognition and segmentation of facial features such as eyebrows, eyes, nose, mouth and ears. The semantic segmentation model is obtained by pre-training a plurality of facial image samples containing facial feature information. This embodiment does not limit the type of semantic segmentation model. For example, the semantic segmentation model can be a segmentation model based on deep learning of convolutional neural network (Convolutional Neural Network, CNN), U-Net model, DeepLab (Deep Label Semantic Image Segmentation, DeepLab) series model or mobile network (Efficient Convolutional Neural Networks for Mobile Vision Applications V2, MobileNetV2) model, etc. Among them, the DeepLab series model includes but is not limited to DeepLabv1, DeepLabv2, DeepLabv3 and DeepLabv3+ models. The semantic segmentation model can also be a model based on the Transformer vision network structure (Vision Transformer, ViT) class, such as the SETR (Semantic Segmentation Transformer) model, the Segmenter model or the SegFormer model, and so on.

[0041] In this another optional embodiment, the gender attribute of the user is identified based on the initial facial feature information contained in the facial image, including: inputting the facial image into a semantic segmentation model for semantic segmentation to obtain a binary image containing the facial feature area; obtaining a local image of the facial feature area based on the target facial image and the binary image, the local image being a partial image of the target facial image; inputting the local image into a feature extraction network layer in the facial recognition model to extract initial facial features to obtain initial facial feature information of the local image; inputting the initial facial feature information of the local image into a gender classification network layer in the facial recognition model to perform gender classification to obtain the gender attribute of the user.

[0042] The binary image of the facial features area refers to a two-dimensional array or image of the same size as the facial features area, which visually presents a black and white image. Each pixel value in a binary image represents the attribution or attribute of the pixel in a certain classification or recognition task. Specifically, a binary image can be used to represent the result of image segmentation, in which each pixel is assigned a probability label, which indicates the possibility that the corresponding pixel belongs to a specific category. These categories with the highest probability are usually used to determine the final category label of each pixel.

[0043] Optionally, the face image is input into a semantic segmentation model for semantic segmentation to obtain a binary image containing facial features, including: inputting the face image into the encoder of the semantic segmentation model, and using the encoder to gradually extract facial features; inputting the facial features into the decoder of the semantic segmentation model, and using the decoder to gradually restore the image resolution to obtain the pixel value of each pixel; based on the pixel reference value of each facial feature area and the pixel value of each pixel, calculating the category probability of the facial feature area to which each pixel belongs, and obtaining a probability map of each pixel in each facial feature area; converting the probability map into a binary image, in which the facial feature area is marked as 1 and other areas are marked as 0. Each pixel is assigned a probability, indicating the possibility that the pixel belongs to a specific category, and the category with the highest probability is usually used to determine the final category label of each pixel.

[0044] More specifically, in the encoder part, the continuous convolution and pooling operations of the semantic segmentation model gradually reduce the spatial dimension of the face image, while increasing the depth of the facial features to capture more abstract facial features, namely high-level abstract features. High-level abstract features in machine learning and computer vision refer to complex and deep feature representations extracted from raw data. These features are not just simple pixel values ​​or basic image attributes, but high-level interpretations and understandings of the data, which can capture deeper patterns and information in the data. In addition, high-level abstract features also have the following characteristics: Hierarchy. High-level abstract features are usually at a higher level in the hierarchy of feature extraction. It is obtained by further abstraction and combination based on low-level features (such as edges, colors, and textures); Complexity: High-level abstract features are more complex than low-level features and can represent more complex data patterns, such as parts, shapes, or overall structures of human faces; Semantics: High-level abstract features tend to have stronger semantics, that is, they can better represent the semantic information of data, such as facial expressions, facial features, etc.; Invariance: High-level abstract features usually have a certain invariance, such as being invariant or robust to transformations such as translation, rotation, and scaling of images; Multi-scale: High-level abstract features may contain information of different scales and can capture local details and global context at the same time; Cross-domain: Some high-level abstract features may be applicable in different fields or tasks. For example, features learned from natural images may also be applicable to medical image analysis; Feature Fusion: In some cases, high-level abstract features may be the fusion result of features of multiple different types or sources to obtain a more comprehensive representation.

[0045] In the decoder part, the semantic segmentation model gradually restores the spatial dimension of the image through upsampling and transposed convolution, while reducing the depth of the facial features to obtain the feature map of the facial features; at the end of the decoder, the feature map is converted into the category probability of each pixel through one or more convolutional layers, which usually involves a convolutional layer with multiple output channels, each channel corresponding to a category probability; the output of the convolutional layer is converted into a probability distribution using Softmax or other activation functions, which can ensure that for each pixel, the sum of the probabilities of all categories is 1; output a probability map, in which each element corresponds to the pixel at the corresponding position in the input face image and contains the probability that the pixel belongs to each category; each pixel in the probability map is binarized according to a determined threshold to obtain a binary image. Among them, if the probability of a pixel belonging to a certain category is higher than the threshold, the pixel is marked as 1 (or the corresponding category label) in the binary image, otherwise it is marked as 0. The threshold usually refers to the global threshold used to convert continuous probabilities into discrete category labels. The threshold is the same for all pixels in the same image. This is because the threshold is a global processing method that determines the category based on the probability distribution of the entire image, rather than setting a threshold for each pixel individually. The specific value of the threshold is pre-set according to actual needs. The threshold can be fixed or dynamically selected based on the performance on the validation set.

[0046] It should be noted that the binary image of the facial features in the present embodiment may be a binary image including the facial features; or each facial feature may correspond to a binary image. The binary image of the face includes the following multiple binary images: a binary image corresponding to the eyebrow area, a binary image corresponding to the eye area, a binary image corresponding to the nose area, a binary image corresponding to the mouth area, and a binary image corresponding to the ear area.

[0047] Optionally, a local image of the facial features is obtained based on the target facial image and the binary image, including: based on the size of the binary image and the facial area included, adjusting the facial image to the same size and direction as the binary image to ensure that the two are aligned; locating the facial features according to the binary image; and cropping the facial image according to the positioning result to obtain a local image of the facial features corresponding to the marked area in the binary image. When locating the facial features according to the binary image, it can be achieved by scanning the non-zero values ​​(representing the facial features) in the binary image. It should be noted that if the facial features are close to the edge of the image, special processing may be required during cropping to ensure the integrity of the local image.

[0048] Optionally, the local image is input into the feature extraction network layer in the face recognition model to extract facial features to obtain facial feature information of the local image, including: first extracting primary features of the local image, such as edges and textures of facial features; gradually combining the primary features into more complex feature representations, which contain initial facial feature information; extracting features according to the feature level to be extracted to obtain facial feature information.

[0049] Optionally, the facial feature information of the partial image is input into the gender classification network layer in the face recognition model for gender classification to obtain the gender attribute of the user, including: performing dimensionality reduction processing on the initial facial feature information to obtain a low-dimensional feature vector of the initial facial feature information; performing gender recognition on the low-dimensional feature vector of the initial facial feature information through the gender classification network layer to obtain the gender attribute of the user. The gender classification network layer of the face recognition model is pre-trained based on the face image sample information set and the gender of the corresponding face image sample. The training process of the gender classification network layer of the face recognition model is as follows: obtain a face image sample information set and the gender of the corresponding image sample; determine the key points of each sample in the face image sample information set, where the key points are the corner points of each facial feature; obtain the initial facial feature information of each sample in the face image sample information set based on each sample and its key points in the face image sample information set; perform dimensionality reduction processing on the initial facial feature information of each sample in the face image sample information set to obtain a low-dimensional feature vector of each sample; train the face recognition model through the low-dimensional feature vector of each sample and its corresponding gender value to obtain a face recognition model. More specifically, when training the face recognition model, the focus is on training the gender classification network layer of the face recognition model to obtain a gender classification network layer with gender recognition function.

[0050] Among them, the initial facial feature information is subjected to dimensionality reduction processing to obtain a low-dimensional feature vector of the initial facial feature information, including: obtaining a feature dimensionality reduction matrix and the initial facial feature information; the initial facial feature information is subjected to dimensionality reduction processing through the feature dimensionality reduction matrix to obtain a low-dimensional feature vector of the initial facial feature information.

[0051] Among them, the gender recognition network layer of the face recognition model may include: a standard female face network layer, a standard male face network layer and a similarity measurement network layer. Based on this, in an optional embodiment, the gender recognition of the low-dimensional feature vector of the initial facial feature information is performed through the gender classification network layer, including: presetting a first similarity threshold of the standard female face model and a second similarity threshold of the standard male face model; obtaining the first similarity between the low-dimensional feature vector of the initial facial feature information and the standard female face model and the second similarity between the low-dimensional feature vector of the initial facial feature information and the standard male face model through the similarity measurement model; judging whether the first similarity exceeds the first threshold; if the first similarity exceeds the first threshold, it is preliminarily judged as a female; otherwise, it is preliminarily judged as a male; judging whether the second similarity exceeds the second threshold; if the second similarity exceeds the second threshold, it is preliminarily judged as a male; otherwise, it is preliminarily judged as a female; if the judgment is consistent, the judgment result is output; if the judgment is inconsistent, an unrecognizable result is output. The gender recognition process using a double verification method can improve the accuracy of the gender recognition result.

[0052] In another optional embodiment, a gender recognition process in the form of identification can be used. Gender recognition is performed on the low-dimensional feature vector of the initial facial feature information through a gender classification network layer, including: presetting a similarity difference threshold; obtaining a first similarity between the low-dimensional feature vector of the initial facial feature information and a standard female face model and a second similarity between the low-dimensional feature vector of the initial facial feature information and a standard male face model through a similarity measurement model; determining whether the absolute value of the difference between the first similarity and the second similarity is less than the similarity difference threshold; if the absolute value is less than the similarity difference threshold, outputting an unrecognizable result; if the absolute value is greater than the similarity difference threshold, outputting the gender type corresponding to the larger similarity value.

[0053] In an optional embodiment, after identifying the gender attribute of the user, a target ID photo template is selected based on the gender attribute of the user and multiple ID photo templates corresponding to the gender attribute of the user, including: selecting multiple ID photo templates corresponding to the gender attribute of the user based on the gender attribute of the user, and sending the multiple ID photo templates to the user's terminal device so that the terminal device can display the multiple ID photo templates to the user; and receiving a selection request sent by the terminal device in response to the user's selection operation, and determining any ID photo template selected by the user as the target ID photo template based on the selection request.

[0054] In another optional embodiment, a target ID photo template is selected based on the user's gender attribute and multiple ID photo templates corresponding to the user's gender attribute, including: obtaining the user's template requirement information; determining multiple ID photo templates corresponding to the user's gender attribute based on the user's gender attribute; and selecting an ID photo template that is compatible with the user's template requirement information from the multiple ID photo templates as the target ID photo template.

[0055] Furthermore, after selecting the target ID photo template, the user's initial facial feature information and the target ID photo can be fused through the image fusion model to obtain a fused ID photo. For ease of description and distinction, the fused ID photo is referred to as the first intermediate ID photo. The image fusion model includes a style adaptation network layer and a feature merging network layer. Based on this, the initial facial feature information and the target ID photo template are input into the image fusion model, and the initial facial feature information is fused into the target ID photo template to obtain the first intermediate ID photo, including: inputting the initial facial feature information and the target ID photo template into the style adaptation network layer in the image fusion model, identifying the style of the initial facial feature information and the target ID photo template, and in the case where the style of the initial facial feature information does not match the style of the target ID photo template, adjusting the style of the initial facial feature information according to the style of the target ID photo template to obtain the target facial feature information; inputting the target facial feature information and the target ID photo template into the feature merging network layer of the image fusion model, identifying the position information of the facial features in the target ID photo template, and fusing the facial feature information in the target ID photo template with the target facial feature information according to the position information of the facial features in the target ID photo to obtain the first intermediate ID photo. This embodiment does not limit the type of fusion model. For example, the fusion model can be an Inswapper model, which is a deep learning model for face replacement function, which can replace one person's face with another person's face in an image or video. The replacement is not a direct replacement of the face image, but a replacement at the face feature level.

[0056] The first intermediate ID photo is obtained by preliminarily fusing the initial facial feature information with the target ID photo template, and the clarity of the facial features and the similarity with the photographed user's facial image are relatively low.

[0057] In order to improve the clarity of the facial features of the first intermediate ID photo, the first intermediate ID photo may be input into an image restoration model to restore the facial region in the first intermediate ID photo to obtain a second intermediate ID photo.

[0058] In some embodiments, a variational autoencoder (VAE) is also introduced. VAE consists of two parts, an encoder and a decoder. The encoder maps the input data to the distribution parameters (usually mean and variance) of a latent space, and the decoder generates data from the distribution of the latent space.

[0059] Based on this, in some embodiments, the image restoration model includes: an image defect detection network layer, an encoder, a restoration network layer, and a decoder. Based on this, the first intermediate ID photo is input into the image restoration model, and the face area in the first intermediate ID photo is restored to obtain the second intermediate ID photo, including: inputting the first intermediate ID photo into the image defect detection network layer of the image restoration model, detecting the defect area of ​​the first intermediate ID photo, and generating a mask image of the defect area when the defect area is detected, the mask image is a binary image used to identify the defect area in the image; further, the face image (the face image initially obtained, and it is believed that the face features contained in the original face image are complete and accurate), the first intermediate ID photo and the mask image are input into the encoder, and the encoder uses a multi-layer convolutional neural network layer to extract the mask image. The incomplete features contained in the local image corresponding to the image in the first intermediate ID photo, and the encoder uses a multi-layer convolutional neural network layer to extract the complete features contained in the local image corresponding to the mask image in the original face image; further, the incomplete features, the complete features, the first intermediate ID photo and the mask image are input into the repair network layer of the image repair model, and the incomplete features of the local image corresponding to the mask image in the first intermediate ID photo are repaired according to the complete features to obtain the repaired features; the repaired features are input into the decoder, and the decoder is used to reconstruct the local face image corresponding to the incomplete area; the reconstructed local face image is fused with the first intermediate ID photo to obtain the second intermediate ID photo.

[0060] It should be noted that if the facial features of the initially acquired facial image are unclear and incomplete, it is necessary to re-acquire the current clearest and most complete facial features to repair the facial image in order to obtain the second intermediate ID photo.

[0061] Among them, the image defect detection network layer can be regarded as a Code Attention module, which can use the attention mechanism to enhance the model's ability to capture information about defective areas. Specifically, in order to enable the model to perceive the relative position relationship between different features, relative position encoding is introduced, which helps the model identify features around the defective area and better understand its structure. Among them, the conditional attention mechanism can adjust the attention distribution according to specific conditions (such as the position and size of the defective area), which enables the model to focus more on features related to the defective area in a targeted manner. The multi-head self-attention mechanism can process information at different positions in parallel, allowing the model to capture information from different representation subspaces at the same time, which helps to better understand the context of the defective area. In addition, the Code Attention module can also establish a connection between the encoder and the decoder, using feature maps of different resolutions to provide more comprehensive contextual information, which helps the model capture details of the defective area at different scales.

[0062] Among them, the features provided by the decoder are multi-scale feature representations, and accordingly, the features after restoration are also multi-scale feature representations. When the decoder is used to generate the second intermediate ID photo, the decoder uses the multi-scale feature representation and the feature sequence generated by the CodeAttention module, and uses a multi-head self-attention mechanism to process information from different representation subspaces in parallel, which helps to capture global context information. The cross-attention layer allows the hidden state of the decoder to interact with the output of the encoder, thereby combining the features extracted by the encoder with the code sequence. The layer normalization and residual connection in the decoder help stabilize the training process and prevent the gradient from disappearing or exploding. The spatial resolution of the feature map is gradually increased through the upsampling layer in the network structure, thereby gradually restoring the details of the image. In addition, the feature map is further refined using a convolutional layer to generate a feature representation of the same size as the original image. Further, the features at different levels are fused using feature fusion technology to enhance the details and quality of the reconstructed image. Finally, the feature map is converted into the final pixel value through a convolutional layer to obtain the second intermediate ID photo. The tanh activation function is usually used to limit the range of pixel values ​​to [-1,1].

[0063] Furthermore, in order to improve the clarity of the ID photo and the similarity with the photographed user's facial image, a Wensheng graph model can be introduced, and the second intermediate ID photo can be further refined based on the Wensheng graph model to obtain the target ID photo.

[0064] The text graph model is used to output an image that conforms to the description of the text information based on the input image and text information after a certain number of iterations. This embodiment does not limit the type of text graph model, for example, it can be an SD model. In addition, when selecting an SD model, a model that has been pre-trained, generalized, adapted to the user's ethnicity, and generates high-definition images can be selected, such as the Stable Diffusion XL (SDXL) model. The SDXL model is a new version of the SD model series, which has made significant improvements and optimizations in image generation quality.

[0065] Among them, the text information refers to at least one prompt word corresponding to the target ID photo template, the user's gender and the user's actual needs. At least one prompt word includes at least a positive prompt word and a reverse prompt word. The positive prompt word is a text description or keyword used to guide the model to generate a specific type or style and high-quality image. The reverse prompt word refers to a text description or keyword used to guide the model to avoid generating certain specific features or elements. The reverse prompt word is opposite to the positive prompt word, and the two work together in the generation process of the model to achieve more accurate control of facial features. In addition, this embodiment does not limit the method for obtaining text information. For example, the text information is stored in association with each ID photo template, that is, each ID photo template is associated with the positive prompt word and reverse prompt word related to the ID photo template.

[0066] In some embodiments, in order to enhance the functionality and flexibility of the Wensheng graph model, a plug-in model may be integrated into the Wensheng graph model. This embodiment does not specifically limit the type of plug-in model. For example, the plug-in model may be an InstantID model, which includes a facial feature constraint model and a facial feature extraction model. The facial feature constraint model may be a Controlnet model, and the facial feature extraction model may be an Ipadapter model. The facial feature constraint model and the facial feature extraction model coexist, that is, the image processing is performed separately through the decoupled cross-attention mechanism, and they will not affect each other.

[0067] Based on the above content, at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template can be further input into the text graph model, and the second intermediate ID photo can be refined according to the at least one prompt word to obtain the target ID photo. Optionally, at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template is input into the Wensheng graph model, and the second intermediate ID photo is refined according to the at least one prompt word to obtain the target ID photo, including: obtaining first relative position information of facial features contained in the target face image of the user; constraining the positions of facial features of the second intermediate ID photo using a facial feature constraint model to obtain second relative position information of the constrained facial features; and, inputting the second intermediate ID photo into a facial feature extraction model to obtain facial feature information of the second intermediate ID photo; determining the resolution of the target ID photo, creating a latent space of the target ID photo according to the resolution, wherein the vector dimension of the latent space is adapted to the feature space dimension of the target ID photo; using an encoder to map the second relative position information of the facial features and the facial feature information of the second intermediate ID photo to the latent space to obtain latent space features of the facial features of the second intermediate ID photo in the latent space; according to at least one prompt word, using a sampler to collect facial feature features adapted to at least one prompt word from the latent space features of the facial features; using a decoder to decode the facial feature feature vector to obtain the target ID photo.

[0068] The above embodiments are mainly aimed at the refined processing of facial features, but the clarity of the skin color and texture of the target ID photo and the similarity between the skin color and texture of the captured user's facial image are poor.

[0069] In order to solve the technical problem that the clarity of the skin color and texture of the target ID photo and the similarity of the skin color and texture with the captured user's face image are poor, in this embodiment, it also includes: inputting the target ID photo into the skin color enhancement and magnification model, amplifying the target ID photo according to the magnification factor of the skin color enhancement and magnification model, and enhancing the skin color pixels of the amplified target ID photo to obtain a first intermediate optimized ID photo; obtaining a first redrawing area of ​​the first intermediate optimized ID photo; inputting the first redrawing area into the facial feature extraction model to obtain facial feature feature information contained in the first redrawing area: Using an encoder, the facial feature information contained in the first redrawn area is mapped to a latent space to obtain latent space features of the facial feature information contained in the first redrawn area in the latent space; based on at least one prompt word, using a sampler, facial feature features that are compatible with at least one prompt word are collected from the latent space features corresponding to the facial feature information contained in the first redrawn area in the latent space; using a decoder, the facial feature features that are compatible with at least one prompt word are collected from the latent space features corresponding to the facial feature information contained in the first redrawn area in the latent space to be decoded to obtain a redrawn second intermediate optimized ID photo.

[0070] Optionally, the target ID photo is input into a skin color enhancement magnification model, the target ID photo is magnified according to the magnification factor of the skin color enhancement magnification model, and the skin color pixels of the magnified target ID photo are enhanced to obtain a first intermediate optimized ID photo, including: obtaining image information of a skin color area of ​​a face in the target ID photo, the image information including a first color vector of each pixel point in the skin color area, the first color vector including a first red component color difference value and a first blue component color difference value; calculating a first skin color center area in the skin color area according to the first red component color difference value and the first blue component color difference value; obtaining a skin color center area of ​​a face in the target ID photo after the skin color enhancement magnification model is magnified; The image information of the color area includes the second color vector of each pixel in the skin color area, and the second color vector includes the second red component color difference value and the second blue component color difference value; according to the second red component color difference value and the second blue component color difference value contained in the second color component, the second skin color center area in the skin color area is calculated; according to the first red component color difference value, the first blue component color difference value, the first skin color center area and the second red component color difference value, the second blue component color difference value, the second skin color center area, the offset of the first color vector and the second color vector of each pixel in the two skin color center areas is calculated; according to the offset, the image is subjected to skin color enhancement processing. Thus, the selection of the skin color center area in the dynamic skin color model is set, so that the human eye's perception of color difference in the selected skin color center area truly reflects the real skin color in the current scene, so that the processing method has a good skin color enhancement effect and has wide applicability.

[0071] In order to improve the clarity of the first intermediate optimized ID photo, a second redrawn area of ​​the second intermediate optimized ID photo can also be obtained; the second redrawn area is input into the facial feature extraction model to obtain the facial feature information contained in the second redrawn area: using an encoder, the facial feature information contained in the second redrawn area is mapped to a latent space to obtain the latent space features of the facial feature information contained in the second redrawn area in the latent space; according to at least one prompt word, using a sampler, facial feature features that are adapted to at least one prompt word are collected from the latent space features corresponding to the facial feature information contained in the second redrawn area in the latent space; using a decoder, the facial feature features that are adapted to at least one prompt word are collected from the latent space features corresponding to the facial feature information contained in the second redrawn area in the latent space to obtain a redrawn optimized ID photo.

[0072] In addition, it should be noted that a sampler can also be introduced in the above embodiments. In image processing, the sampler can select or extract a specific part from an image or a set of images. When processing face images, the DPM++2M Karras sampler can be selected. Samplers have a variety of functions, such as data sampling: in machine learning, samplers are used to select a portion of data from a large data set for training or verification to improve the efficiency and performance of the model algorithm; feature point sampling: in image feature extraction, samplers can be used to select key points or feature points, which are very important in image recognition and matching; image region selection: in image segmentation or object detection tasks, samplers can be used to select regions of interest for more detailed processing; image fusion: in image synthesis or fusion, samplers may be used to select content from different images and merge them into a new image; image super-resolution: in image super-resolution technology, samplers are used to extract feature information from low-resolution images to generate higher-resolution images; image denoising: samplers can select key information in the image during the denoising process to reduce the impact of noise on image quality; image style transfer: in style transfer tasks, samplers can be used to select and apply features of a specific style to the target image; image generation: when using a generative model, samplers may be used to sample from the latent space to generate a new image.

[0073] At this point, the ID photo generation method is completed. Figure 1b Shown is a comparison diagram between the initially acquired facial image and the finally obtained optimized ID photo (the left side is the initially acquired facial image, and the right side is the optimized ID photo). It can be seen that the technical solutions provided by the above-mentioned embodiments of the present application, given a facial image of a user with low clarity and a tilted face, can generate a clear ID photo of the user in one click. There is no need to collect multiple facial images of the user before image processing to obtain a target ID photo with high clarity, and the overall similarity between the target ID photo and the original photo can also be improved.

[0074] Figure 2 The following is a flow chart of an image processing method provided by an exemplary embodiment of the present application. Figure 2 ,include:

[0075] 201. Acquire a facial image of a target object, where the facial image includes initial facial feature information of the target object;

[0076] 202. Identify the gender attribute of the target object according to the initial facial feature information contained in the facial image;

[0077] 203. Select a target photo template according to the gender attribute of the target object and a plurality of photo templates corresponding to the gender attribute of the target object;

[0078] 204. Inputting the initial facial feature information and the target ID photo template into the image fusion model, and fusing the initial facial feature information into the target photo template to obtain a first intermediate photo;

[0079] 205. Input the first intermediate ID photo into the image restoration model, and perform restoration on the facial area in the first intermediate photo to obtain a second intermediate photo;

[0080] 206. Input at least one prompt word corresponding to the second intermediate photo and the target photo template into the Wensheng graph model, and perform fine processing on the second intermediate photo according to the at least one prompt word to obtain the target photo.

[0081] The detailed implementation methods and beneficial effects of each step provided in the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated in detail here.

[0082] Figure 3 This is a schematic diagram of the structure of the ID photo generation device provided by the exemplary embodiment of the present application. Figure 3 As shown, including:

[0083] An acquisition module 31 is used to acquire a user's face image, where the face image contains the user's initial facial features information;

[0084] The recognition module 32 is used to recognize the gender attribute of the user based on the initial facial features information contained in the face image;

[0085] A selection module 33, configured to select a target ID photo template according to the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user;

[0086] A fusion module 34 is used to input the initial facial feature information and the target ID photo template into the image fusion model, and fuse the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo;

[0087] The restoration module 35 is used to input the first intermediate ID photo into the image restoration model and restore the face area in the first intermediate ID photo to obtain a second intermediate ID photo;

[0088] The optimization module 36 is used to input at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into the text graph model, and perform fine processing on the second intermediate ID photo according to the at least one prompt word to obtain the target ID photo.

[0089] In an optional embodiment, when the recognition module 32 identifies the gender attribute of the user based on the initial facial feature information contained in the facial image, it is specifically used to: input the target facial image into the feature extraction network layer in the face recognition model to extract the facial features to obtain the facial feature information; input the facial feature information into the gender classification network layer of the face recognition model to perform gender classification to obtain the gender attribute of the user.

[0090] In another optional embodiment, when the recognition module 32 identifies the gender attribute of the user based on the facial feature information contained in the facial image, it is specifically used to: input the facial image into a semantic segmentation model for semantic segmentation to obtain a binary image containing the facial feature area; obtain a local image of the facial feature area based on the target facial image and the binary image, the local image being a partial image of the target facial image; input the local image into the feature extraction network layer in the facial recognition model to extract facial features to obtain facial feature information of the local image; input the facial feature information of the local image into the gender classification network layer in the facial recognition model for gender classification to obtain the gender attribute of the user.

[0091] In some embodiments, when the selection module 33 selects the target ID photo template according to the user's gender attribute and the multiple ID photo templates corresponding to the user's gender attribute, it is specifically used to: select multiple ID photo templates corresponding to the user's gender attribute according to the user's gender attribute, and send the multiple ID photo templates to the user's terminal device so that the terminal device can display the multiple ID photo templates to the user; and receive the selection request sent by the terminal device in response to the user's selection operation, and determine any ID photo template selected by the user as the target ID photo template according to the selection request. Alternatively, obtain the user's template requirement information; determine multiple ID photo templates corresponding to the user's gender attribute according to the user's gender attribute; and select the ID photo template that is adapted to the user's template requirement information from the multiple ID photo templates as the target ID photo template.

[0092] In some embodiments, the image fusion model includes a style adaptation network layer and a feature merging network layer. When the fusion module 34 inputs the initial facial feature information and the target ID photo template into the image fusion model, and fuses the initial facial feature information into the target ID photo template to obtain the first intermediate ID photo, it is specifically used to: input the initial facial feature information into the style adaptation network layer in the image fusion model, identify the style of the initial facial feature information and the target ID photo template, and when the style of the initial facial feature information does not match the style of the target ID photo template, adjust the style of the initial facial feature information according to the style of the target ID photo template to obtain the target facial feature information; input the target facial feature information and the target ID photo template into the feature merging network layer of the image fusion model, identify the position information of the facial features in the target ID photo template, and fuse the facial feature information in the target ID photo template with the target facial feature information according to the position information of the facial features, that is, correct the facial feature information in the target ID photo template to the target facial feature information, so as to obtain the first intermediate ID photo.

[0093] In some embodiments, the repair module 35 inputs the first intermediate ID photo into the image repair model, repairs the face area in the first intermediate ID photo to obtain the second intermediate ID photo, and is specifically used to: input the first intermediate ID photo into the image defect detection network layer of the image repair model, detect the defect area of ​​the first intermediate ID photo, and when the defect area is detected, generate a mask image of the defect area; input the user's face image, the first intermediate ID photo and the mask image into the encoder, and use a multi-layer convolutional neural network layer to extract the local image corresponding to the mask image in the first intermediate ID photo. The incomplete facial features and the complete facial features contained in the local image corresponding to the facial image; input the incomplete facial features, the complete facial features, the first intermediate ID photo and the mask image into the repair network layer of the image repair model, and repair the incomplete facial features of the local image corresponding to the first intermediate ID photo according to the complete facial features to obtain the repaired facial features; input the repaired facial features into the decoder, and use the decoder to reconstruct the local facial image corresponding to the incomplete area; fuse the reconstructed local facial image with the first intermediate ID photo to obtain the second intermediate ID photo.

[0094] In this embodiment, the Wensheng graph model is integrated with a plug-in model, and the plug-in model includes a facial feature constraint model and a facial feature extraction model. Based on this, when the optimization module 36 inputs at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into the Wensheng graph model, and performs fine processing on the second intermediate ID photo according to the at least one prompt word to obtain the target ID photo, it is specifically used to: obtain the first relative position information of the facial features contained in the target face image of the user; use the facial feature constraint model to constrain the positions of the facial features of the second intermediate ID photo to obtain the second relative position information of the facial features after the constraints; and input the second intermediate ID photo into the facial feature extraction model to obtain the facial features of the second intermediate ID photo. feature information; determining a resolution of the target ID photo, creating a latent space of the target ID photo according to the resolution, wherein the vector dimension of the latent space is adapted to the feature space dimension of the target ID photo; using an encoder, mapping the second relative position information of the facial features and the facial feature feature information of the second intermediate ID photo to the latent space, and obtaining latent space features of the facial features of the second intermediate ID photo in the latent space; according to the at least one prompt word, using a sampler, collecting facial feature features adapted to the at least one prompt word from the latent space features of the facial features; using a decoder, decoding the facial feature feature vector to obtain the target ID photo.

[0095] Furthermore, the optimization module 36 is also used to input the target ID photo into the skin color enhancement magnification model, magnify the target ID photo according to the magnification factor of the skin color enhancement magnification model, and enhance the skin color pixels of the enlarged target ID photo to obtain a first intermediate optimized ID photo; obtain a first redrawing area of ​​the first intermediate optimized ID photo; input the first redrawing area into the facial feature extraction model to obtain facial feature information contained in the first redrawing area: use an encoder to map the facial feature information contained in the first redrawing area to the latent space to obtain latent space features of the facial feature information contained in the first redrawing area in the latent space; according to the at least one prompt word, use a sampler to collect facial feature features that are compatible with the at least one prompt word from the latent space features corresponding to the facial feature information contained in the first redrawing area in the latent space; use a decoder to decode the facial feature features that are compatible with the at least one prompt word from the latent space features corresponding to the latent space of the facial feature information contained in the first redrawing area to obtain a redrawn second intermediate optimized ID photo.

[0096] Furthermore, the optimization module 36 is also used to obtain a second redrawn area of ​​the second intermediate optimized ID photo; input the second redrawn area into the facial feature extraction model to obtain the facial feature information contained in the second redrawn area: use an encoder to map the facial feature information contained in the second redrawn area to the latent space to obtain the latent space features of the facial feature information contained in the second redrawn area in the latent space; according to the at least one prompt word, use a sampler to collect facial feature features that are compatible with the at least one prompt word from the latent space features corresponding to the facial feature information contained in the second redrawn area in the latent space; use a decoder to decode the facial feature features that are compatible with the at least one prompt word collected from the latent space features corresponding to the latent space in the latent space of the facial feature information contained in the second redrawn area to obtain the redrawn optimized ID photo.

[0097] The detailed implementation and beneficial effects of each module provided in the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0098] Figure 4 The schematic diagram of the structure of the image processing device provided by the exemplary embodiment of the present application is shown in FIG. Figure 4 As shown, including:

[0099] An acquisition module 41 is used to acquire a facial image of a target object, wherein the facial image includes initial facial feature information of the target object;

[0100] The recognition module 42 is used to recognize the gender attribute of the target object according to the initial facial features information contained in the facial image;

[0101] A selection module 43, configured to select a target photo template according to the gender attribute of the target object and a plurality of photo templates corresponding to the gender attribute of the target object;

[0102] A fusion module 44 is used to input the initial facial feature information and the target ID photo template into the image fusion model, and fuse the initial facial feature information into the target photo template to obtain a first intermediate photo;

[0103] The restoration module 45 is used to input the first intermediate ID photo into the image restoration model and restore the facial area in the first intermediate photo to obtain a second intermediate photo;

[0104] The optimization module 46 is used to input at least one prompt word corresponding to the second intermediate photo and the target photo template into the Wensheng graph model, and perform fine processing on the second intermediate photo according to the at least one prompt word to obtain the target photo.

[0105] The detailed implementation and beneficial effects of each module provided in the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0106] Figure 5 The following is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Figure 5 As shown, it includes: a memory 50a and a processor 50b; the memory 50a is used to store a computer program; the processor 50b is coupled to the memory 50a and is used to execute the computer program to implement the following steps:

[0107] A facial image of a user is obtained, the facial image including initial facial feature information of the user; the gender attribute of the user is identified based on the initial facial feature information included in the facial image; a target ID photo template is selected based on the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user; the initial facial feature information and the target ID photo template are input into an image fusion model, and the initial facial feature information is fused into the target ID photo template to obtain a first intermediate ID photo; the first intermediate ID photo is input into an image restoration model, and the facial region in the first intermediate ID photo is restored to obtain a second intermediate ID photo; at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template is input into a text map model, and the second intermediate ID photo is refined according to the at least one prompt word to obtain a target ID photo.

[0108] In an optional embodiment, when the processor 50b identifies the gender attribute of the user based on the initial facial feature information contained in the facial image, it is specifically used to: input the target facial image into the feature extraction network layer in the face recognition model to extract the facial features to obtain the facial feature information; input the facial feature information into the gender classification network layer of the face recognition model to perform gender classification to obtain the gender attribute of the user.

[0109] In another optional embodiment, when the processor 50b identifies the gender attribute of the user based on the facial feature information contained in the facial image, it is specifically used to: input the facial image into a semantic segmentation model for semantic segmentation to obtain a binary image containing the facial feature area; obtain a local image of the facial feature area based on the target facial image and the binary image, the local image being a partial image of the target facial image; input the local image into the feature extraction network layer in the facial recognition model to extract facial features to obtain facial feature information of the local image; input the facial feature information of the local image into the gender classification network layer in the facial recognition model for gender classification to obtain the gender attribute of the user.

[0110] In some embodiments, when the processor 50b selects the target ID photo template according to the user's gender attribute and the multiple ID photo templates corresponding to the user's gender attribute, it is specifically used to: select multiple ID photo templates corresponding to the user's gender attribute according to the user's gender attribute, and send the multiple ID photo templates to the user's terminal device so that the terminal device can display the multiple ID photo templates to the user; and receive a selection request sent by the terminal device in response to the user's selection operation, and determine any ID photo template selected by the user as the target ID photo template according to the selection request. Alternatively, obtain the user's template requirement information; determine multiple ID photo templates corresponding to the user's gender attribute according to the user's gender attribute; and select an ID photo template that is adapted to the user's template requirement information from the multiple ID photo templates as the target ID photo template.

[0111] In some embodiments, the image fusion model includes a style adaptation network layer and a feature merging network layer. When the processor 50b inputs the facial feature information and the target ID photo template into the image fusion model, and fuses the initial facial feature information into the target ID photo template to obtain the first intermediate ID photo, it is specifically used to: input the initial facial feature information into the style adaptation network layer in the image fusion model, identify the style of the initial facial feature information and the target ID photo template, and when the style of the initial facial feature information does not match the style of the target ID photo template, adjust the style of the initial facial feature information according to the style of the target ID photo template to obtain the target facial feature information; input the target facial feature information and the target ID photo template into the image fusion model, identify the position of the facial features in the target ID photo template, and correct the facial feature information in the target ID photo template to the target facial feature information according to the position information of the facial features, so as to obtain the first intermediate ID photo.

[0112] In some embodiments, the processor 50b inputs the first intermediate ID photo into the image restoration model, and restores the face area in the first intermediate ID photo to obtain the second intermediate ID photo, and is specifically used to: input the first intermediate ID photo into the image defect detection network layer of the image restoration model, detect the defect area of ​​the first intermediate ID photo, and generate a mask image of the defect area when the defect area is detected; input the user's face image, the first intermediate ID photo and the mask image into the encoder, and use a multi-layer convolutional neural network layer to extract the defect area contained in the local image corresponding to the mask image in the first intermediate ID photo. The incomplete facial features and the complete facial features contained in the local image corresponding to the facial image; the incomplete facial features, the complete facial features, the first intermediate ID photo and the mask image are input into the repair network layer of the image repair model, and the incomplete facial features of the local image corresponding to the mask image in the first intermediate ID photo are repaired according to the complete facial features to obtain the repaired facial features; the repaired facial features are input into the decoder, and the decoder is used to reconstruct the local facial image corresponding to the incomplete area; the reconstructed local facial image is fused with the first intermediate ID photo to obtain the second intermediate ID photo.

[0113] In this embodiment, the Wensheng graph model is integrated with a plug-in model, and the plug-in model includes a facial feature constraint model and a facial feature extraction model. Based on this, after obtaining the second intermediate ID photo, the processor 50b is used to input at least one prompt word corresponding to the second intermediate ID photo and the target ID photo template into the Wensheng graph model, and perform fine processing on the second intermediate ID photo according to the at least one prompt word to obtain the target ID photo, specifically for: obtaining the first relative position information of the facial features contained in the target face image of the user; using the facial feature constraint model to constrain the positions of the facial features of the second intermediate ID photo to obtain the second relative position information of the facial features after the constraints; and inputting the second intermediate ID photo into the facial feature extraction model to obtain the second intermediate ID photo. feature information of facial features of the ID photo; determining the resolution of the target ID photo, creating a latent space of the target ID photo according to the resolution, wherein the vector dimension of the latent space is adapted to the feature space dimension of the target ID photo; using an encoder, mapping the second relative position information of the facial features and the feature information of the facial features of the second intermediate ID photo to the latent space, and obtaining the latent space features of the facial features of the second intermediate ID photo in the latent space; according to the at least one prompt word, using a sampler, collecting the facial features features adapted to the at least one prompt word from the latent space features of the facial features; using a decoder, decoding the facial features feature vector to obtain the target ID photo.

[0114] Furthermore, the processor 50b is also used to input the target ID photo into the skin color enhancement magnification model, magnify the target ID photo according to the magnification factor of the skin color enhancement magnification model, and enhance the skin color pixels of the enlarged target ID photo to obtain a first intermediate optimized ID photo; obtain a first redrawing area of ​​the first intermediate optimized ID photo; input the first redrawing area into the facial feature extraction model to obtain facial feature information contained in the first redrawing area: using an encoder, map the facial feature information contained in the first redrawing area to the latent space to obtain the latent space features of the facial feature information contained in the first redrawing area in the latent space; according to the at least one prompt word, using a sampler, collect facial feature features that are compatible with the at least one prompt word from the latent space features corresponding to the facial feature information contained in the first redrawing area in the latent space; using a decoder, decode the facial feature features that are compatible with the at least one prompt word from the latent space features corresponding to the latent space of the facial feature information contained in the first redrawing area to obtain a redrawn second intermediate optimized ID photo.

[0115] Furthermore, the processor 50b is also used to obtain a second redrawn area of ​​the second intermediate optimized ID photo; input the second redrawn area into the facial feature extraction model to obtain the facial feature information contained in the second redrawn area: using an encoder, map the facial feature information contained in the second redrawn area to the latent space to obtain the latent space features of the facial feature information contained in the second redrawn area in the latent space; according to the at least one prompt word, using a sampler, collect facial feature features that are adapted to the at least one prompt word from the latent space features corresponding to the facial feature information contained in the second redrawn area in the latent space; using a decoder, decode the facial feature features that are adapted to the at least one prompt word collected from the latent space features corresponding to the latent space of the facial feature information contained in the second redrawn area to obtain the redrawn optimized ID photo.

[0116] In addition, based on the electronic device with the same structure, the processor 50b is further configured to perform the following steps:

[0117] A facial image of a target object is obtained, wherein the facial image includes initial facial feature information of the target object; the gender attribute of the target object is identified based on the initial facial feature information included in the facial image; a target photo template is selected based on the gender attribute of the target object and a plurality of photo templates corresponding to the gender attribute of the target object; the initial facial feature information and a target ID photo template are input into an image fusion model, and the initial facial feature information is fused into the target photo template to obtain a first intermediate photo; the first intermediate ID photo is input into an image restoration model, and the facial area in the first intermediate photo is restored to obtain a second intermediate photo; the second intermediate photo and at least one prompt word corresponding to the target photo template are input into a Vincent graph model, and the second intermediate photo is refined according to the at least one prompt word to obtain a target photo.

[0118] Furthermore, if Figure 5 As shown, the electronic device also includes: a communication component 50c, a display 50d, a power component 50e, an audio component 50f and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0119] The detailed implementation and beneficial effects of the electronic device provided in the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0120] An exemplary embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to implement the steps in the above-mentioned method embodiments.

[0121] An exemplary embodiment of the present application further provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above-mentioned method embodiments.

[0122] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0123] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0124] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0125] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0126] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0127] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0128] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0130] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0131] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0132] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0133] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0134] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for generating a certificate photo, characterized in that: include: Acquire a facial image of a user, wherein the facial image includes initial facial feature information of the user; Identifying the gender attribute of the user according to the initial facial feature information contained in the facial image; Selecting a target ID photo template according to the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user; Inputting the initial facial feature information and the target ID photo template into an image fusion model, and fusing the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo; Inputting the first intermediate ID photo into an image restoration model to restore the face region in the first intermediate ID photo to obtain a second intermediate ID photo; At least one prompt word corresponding to the second intermediate ID photo and the target ID photo template is input into the text graph model, and the second intermediate ID photo is refined according to the at least one prompt word to obtain the target ID photo.

2. The method according to claim 1, characterized in that Identifying the gender attribute of the user according to the initial facial feature information contained in the facial image includes: Inputting the target face image into the feature extraction network layer in the face recognition model to extract the facial features to obtain the facial feature information; The facial feature information is input into the gender classification network layer of the face recognition model for gender classification to obtain the gender attribute of the user.

3. The method according to claim 1, characterized in that Identifying the gender attribute of the user according to the facial features information contained in the facial image includes: Inputting the face image into a semantic segmentation model for semantic segmentation to obtain a binary image containing facial features; Obtaining a partial image of a facial feature region according to the target facial image and the binary image, wherein the partial image is a partial image of the target facial image; Inputting the partial image into a feature extraction network layer in a face recognition model to extract facial features, so as to obtain facial feature information of the partial image; The facial feature information of the partial image is input into the gender classification network layer in the face recognition model for gender classification to obtain the gender attribute of the user.

4. The method according to claim 1, characterized in that: Selecting a target ID photo template according to the gender attribute of the user and a plurality of ID photo templates corresponding to the gender attribute of the user includes: Selecting, according to the gender attribute of the user, a plurality of ID photo templates corresponding to the gender attribute of the user, and sending the plurality of ID photo templates to a terminal device of the user, so that the terminal device displays the plurality of ID photo templates to the user; and receiving a selection request sent by the terminal device in response to a selection operation of the user, and determining, according to the selection request, any ID photo template selected by the user as a target ID photo template; or, Acquire the template requirement information of the user; determine a plurality of ID photo templates corresponding to the gender attribute of the user according to the gender attribute of the user; and select an ID photo template that is adapted to the template requirement information of the user from the plurality of ID photo templates as the target ID photo template.

5. The method according to claim 1, characterized in that The image fusion model includes a style adaptation network layer and a feature merging network layer, inputs the facial feature information and the target ID photo template into the image fusion model, fuses the initial facial feature information into the target ID photo template to obtain a first intermediate ID photo, including: Inputting the initial facial feature information into the style adaptation network layer in the image fusion model, identifying the styles of the initial facial feature information and the target ID photo template, and in the case where the style of the initial facial feature information does not match the style of the target ID photo template, adjusting the style of the initial facial feature information according to the style of the target ID photo template to obtain the target facial feature information; The target facial feature information and the target ID photo template are input into the image fusion model, the positions of the facial features in the target ID photo template are identified, and based on the position information of the facial features, the facial feature information in the target ID photo template is corrected to correspond to the target facial feature information to obtain a first intermediate ID photo.

6. The method according to claim 1, characterized in that Inputting the first intermediate ID photo into an image restoration model and restoring a face region in the first intermediate ID photo to obtain a second intermediate ID photo, including: Inputting the first intermediate ID photo into an image defect detection network layer of an image restoration model, detecting a defect area on the first intermediate ID photo, and generating a mask image of the defect area when a defect area is detected; Inputting the first intermediate ID photo and the mask image into the restoration network layer of the image restoration model, and restoring the mask image according to the first intermediate ID photo to obtain the first and fourth intermediate ID photos after preliminary restoration; Inputting the first and fourth intermediate ID photos after preliminary restoration into the face detection network layer of the image restoration model, and identifying the face region of the first and fourth intermediate ID photos after preliminary restoration to obtain the face images of the first and fourth intermediate ID photos after preliminary restoration; Inputting the face images of the first and fourth intermediate ID photos after preliminary restoration into the face reconstruction network layer of the image restoration model to obtain a reconstructed face image; The reconstructed facial image is pasted back into the first and fourth intermediate ID photos after the preliminary restoration, and the first and fourth intermediate ID photos after the preliminary restoration with the reconstructed facial image pasted back are super-resolution processed through the image super-resolution network layer of the high-definition restoration model to obtain a second intermediate ID photo.

7. The method according to claim 1, characterized in that The Wensheng graph model is integrated with a plug-in model, and the plug-in model includes a facial feature constraint model; After obtaining the second intermediate ID photo, the method further includes: obtaining first relative position information of facial features contained in the target facial image of the user; constraining the positions of the facial features of the second intermediate ID photograph using the facial feature constraint model to obtain second relative position information of the facial features after being constrained; and inputting the second intermediate ID photograph into the facial feature extraction model to obtain facial feature information of the second intermediate ID photograph; Determine a resolution of the target ID photo, and create a latent space of the target ID photo according to the resolution, wherein the vector dimension of the latent space is adapted to the feature space dimension of the target ID photo; Mapping the second relative position information of the facial features and the facial feature information of the second intermediate ID photo to the latent space using an encoder to obtain latent space features of the facial features of the second intermediate ID photo in the latent space; According to the at least one prompt word, using a sampler, collecting facial features that are adapted to the at least one prompt word from the latent space features of the facial features; The facial feature vector is decoded by using a decoder to obtain the target ID photo.

8. The method according to claim 7, characterized in that Also includes: Inputting the target ID photo into a skin color enhancement magnification model, magnifying the target ID photo according to the magnification factor of the skin color enhancement magnification model, and enhancing the skin color pixels of the magnified target ID photo to obtain a first intermediate optimized ID photo; Acquire a first redrawing area of ​​the first intermediate optimized ID photo; Inputting the first redrawn area into the facial features extraction model to obtain facial features information contained in the first redrawn area: Mapping the facial feature information contained in the first redrawn area to the latent space using an encoder to obtain latent space features of the facial feature information contained in the first redrawn area in the latent space; According to the at least one prompt word, using a sampler, collecting facial features that are adapted to the at least one prompt word from the facial feature information contained in the first redrawing area in the latent space features corresponding to the latent space; Using a decoder, the facial features information contained in the first redrawn area is collected from the latent space features corresponding to the latent space and the facial features adapted to the at least one prompt word are decoded to obtain a redrawn second intermediate optimized ID photo.

9. The method according to claim 8, characterized in that Also includes: Acquire a second redrawing area of ​​the second intermediate optimized ID photo; Input the second redrawn area into the facial feature extraction model to obtain facial feature information contained in the second redrawn area: Mapping the facial feature information contained in the second redrawn area to the latent space using an encoder to obtain latent space features of the facial feature information contained in the second redrawn area in the latent space; According to the at least one prompt word, using a sampler, collecting facial features that are adapted to the at least one prompt word from the facial feature information contained in the second redrawing area in the latent space features corresponding to the latent space; The decoder is used to decode the facial feature information contained in the second redrawn area and the facial features matched with the at least one prompt word in the latent space features corresponding to the latent space to obtain a redrawn optimized ID photo.

10. An image processing method, characterized in that: include: Acquire a facial image of a target object, wherein the facial image includes initial facial feature information of the target object; Identifying the gender attribute of the target object according to the initial facial feature information contained in the facial image; Selecting a target photo template according to the gender attribute of the target object and a plurality of photo templates corresponding to the gender attribute of the target object; Inputting the initial facial feature information and the target ID photo template into an image fusion model, and fusing the initial facial feature information into the target photo template to obtain a first intermediate photo; Inputting the first intermediate ID photo into an image restoration model, and restoring the facial area in the first intermediate photo to obtain a second intermediate photo; At least one prompt word corresponding to the second intermediate photo and the target photo template is input into the Wensheng graph model, and the second intermediate photo is refined according to the at least one prompt word to obtain the target photo.

11. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program in the memory to implement the steps in the method according to any one of claims 1 to 9 or claim 10.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 9 or claim 10.

13. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 9 or claim 10.

Citation Information

Patent Citations

  • Method and device for estimating face score in picture

    CN105718869A

  • Method, system and equipment for repairing old portrait photo and storage medium

    CN116823653A

  • Image generation method and device, electronic equipment and storage medium

    CN117474748A

  • Image generation method and device

    CN117746487A

  • Image face changing method and device, equipment and medium

    CN118261782A

Cited By

  • Personalized portrait generation and style conversion method and device and medium

    CN119810241A

  • Identification photo generation method and device and electronic equipment

    CN119963692A

  • Identification photo generation method and electronic equipment

    CN120526003A

  • Image duplicate checking method and device, storage medium and program product

    CN121191210A

  • Image duplicate detection method, device, storage medium and program product

    CN121191210B