A method for generating an ID photo and an electronic device

Abstract

The application discloses a kind of certificate photo generation method and electronic equipment, relate to image processing field, after obtaining original user image and original template image (certificate photo template) to original user image, according to original user image and image deformation algorithm, original template image is carried out image deformation, obtain target template image for original user image and meet one or more of face key point constraint, light shadow consistency constraint and face structure constraint, only then original user image and target template image are input into trained image generation model (obtained by optimizing training to diffusion algorithm), according to the different processing weights corresponding to face area and non-face area, the corresponding area is processed, to ensure that the target certificate photo obtained is more natural, real in vision, reliably meet the specification requirement of actual scene to certificate photo, and reduce the calculation time and cost, meet large-scale certificate photo batch generation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a method for generating an ID photo and an electronic device. BACKGROUND

[0002] With the increasing demand for digital and intelligent services in society, the demand for ID photos is also showing a trend of diversification and personalization. However, traditional ID photo shooting usually requires users to go to designated photo studios or related institutions and also requires compliance with specific shooting requirements such as background, clothing, posture, etc., which increases the complexity and cost of ID photo shooting and affects user experience and convenience. SUMMARY

[0003] In view of the above problems, the present application provides the following solutions:

[0004] The first aspect of the present application provides a method for generating an ID photo, comprising:

[0005] obtaining an original user image and an original template image corresponding to the original user image; the original template image is from an ID photo template library;

[0006] deforming the original template image according to the original user image and an image deformation algorithm to obtain a target template image corresponding to the original user image;

[0007] inputting the original user image and the target template image into a trained image generation model, processing corresponding regions between the original user image and the target template image according to different processing weights of face regions and non-face regions, and obtaining a target ID photo;

[0008] wherein the image deformation algorithm is obtained by constraint optimization on a thin plate spline (TPS) deformation algorithm, and the constraint includes one or more of a face key point constraint, a light shadow consistency constraint, and a face structure constraint;

[0009] The image generation model is obtained by optimizing and training a diffusion algorithm.

[0010] Optionally, the deforming the original template image according to the original user image and the image deformation algorithm to obtain the target template image corresponding to the original user image comprises:

[0011] detecting face key points of the original template image to obtain a plurality of original face key point positions, and detecting face key points of the original user image to obtain a plurality of user face key point positions;

[0012] The user face key point positions are dynamically matched to obtain a plurality of target face key point positions;

[0013] According to the plurality of original face key point positions and the plurality of target face key point positions, a face key region of the original template image is nonlinearly deformed by an image deformation algorithm, and a face non-key region of the original template image is smoothly transitioned, and a deformed template image satisfying the face key point constraint is obtained through bilinear interpolation calculation;

[0014] According to at least one of the light and shadow consistency constraint and the face structure constraint, the deformed template image is processed to obtain a target template image for the original user image.

[0015] Optionally, according to the plurality of original face key point positions and the plurality of target face key point positions, the face key region of the original template image is nonlinearly deformed by the image deformation algorithm, and the face non-key region of the original template image is smoothly transitioned, including:

[0016] The deformation weight corresponding to each of different face key regions and face non-key regions is obtained;

[0017] According to the deformation weight, the plurality of original face key point positions are mapped to the target face key point positions of the corresponding face key points to realize nonlinear deformation of the face key region of the original template image and smooth transition of the face non-key region, so as to minimize the distance between the original face key point positions and the target face key point positions corresponding to the same face key point;

[0018] The deformation weight corresponding to the face key point is greater than the deformation weight corresponding to the face non-key point; the deformation weight corresponding to the face key point with a higher degree of influence on the quality of the ID photo is greater.

[0019] Optionally, according to the light and shadow consistency constraint, the deformed template image is processed, including:

[0020] The first light and shadow intensity of the original template image and the second light and shadow intensity of the deformed template image are obtained;

[0021] According to the difference between the first light and shadow intensity and the second light and shadow intensity of the same pixel point, a light and shadow residual error between the original template image and the deformed template image is obtained;

[0022] According to the light and shadow residual error and a light and shadow constraint weight, the deformed template image is smoothly processed so that the light and shadow effect of the processed template image is consistent with the light and shadow effect of the original template image.

[0023] Optionally, according to the face structure constraint, the processed template image is processed, including:

[0024] determining a face symmetry line in the deformed template image;

[0025] performing point position mirror transformation on the deformed template image according to the face symmetry line and a symmetry constraint weight, so that the face key points in the deformed template image are left-right symmetrical about the face symmetry line.

[0026] Optionally, the input of the original user image and the target template image into the trained image generation model processes corresponding regions between the original user image and the target template image according to different processing weights corresponding to face regions and non-face regions to obtain a target ID photo, including:

[0027] inputting the original user image and the target template image into a trained image fusion model, and fusing and adding noise to face regions and non-face regions of the original user image and the target template image respectively according to a fusion weight and a dynamic face mask to obtain a fusion and noise added image;

[0028] restoring face regions and non-face regions of the fusion and noise added image respectively according to a generation weight and the dynamic face mask to obtain a target ID photo.

[0029] Optionally, the fusion weight and the generation weight are the same dynamic weighting coefficient, and the method for obtaining the dynamic weighting coefficient includes:

[0030] performing feature extraction on the face regions of the original user image to obtain a first feature vector, and performing feature extraction on the target template image to obtain a second feature vector;

[0031] performing similarity calculation on the first feature vector and the second feature vector to obtain an image similarity;

[0032] matching the positions of face key points between the original user image and the target template image to obtain a face key point matching degree;

[0033] weighting and summing the image similarity and the face key point matching degree, and performing normalization processing to obtain a dynamic weighting coefficient for the original user image and the target template image.

[0034] Optionally, the matching of the positions of face key points between the original user image and the target template image to obtain a face key point matching degree includes:

[0035] aligning a face center point of the original user image with the target template image;

[0036] calculating a relative average offset between the aligned original user image and the target template image to obtain a face key point matching degree between the original user image and the target template image.

[0037] Optionally, the face region and the non-face region of the original user image and the target template image are fused and added with noise according to the fusion weight and the dynamic face mask to obtain a fused and added noise image, and the face region and the non-face region of the fused and added noise image are respectively restored according to the generation weight and the dynamic face mask to obtain a target ID photo, including:

[0038] fusing the original user image and the target template image according to the dynamic weighting coefficient to obtain a fused image;

[0039] adding noise to the face region of the fused image and gradually adding noise to the non-face region of the target template image according to the dynamic face mask, and fusing the added noise face region and the added noise non-face region to obtain a fused and added noise image;

[0040] gradually denoising the face region and the non-face region in the fused and added noise image according to the first feature vector, the second feature vector and the dynamic weighting coefficient to obtain a target ID photo.

[0041] Optionally, in the process of obtaining the image generation model by optimizing and training the diffusion algorithm:

[0042] obtaining a feature consistency loss according to sample feature vectors of a sample fused and added noise image, a sample user image and a sample template image;

[0043] obtaining a light and shadow consistency loss according to sample light and shadow intensities of the sample fused and added noise image and the sample template image;

[0044] obtaining an adversarial loss between a predicted ID photo output by an initial image generation model and a standard ID photo according to a discriminator network;

[0045] obtaining a reconstruction loss of the predicted ID photo according to the predicted ID photo output by the initial image generation model and the standard ID photo;

[0046] obtaining a predicted total loss according to the feature consistency loss, the light and shadow consistency loss, the adversarial loss, the reconstruction loss and respective loss weights;

[0047] training the initial image generation model by minimizing the predicted total loss to obtain an image generation model.

[0048] The second aspect of the present application provides an electronic device, comprising at least one communication element, at least one memory and at least one processor, wherein:

[0049] The communication element is configured to receive an original user image and an original template image corresponding to the original user image, wherein the original template image is from a template library of ID photos;

[0050] The memory is configured to store a plurality of computer program instructions;

[0051] The processor is configured to load and execute the computer program instructions to implement the following steps:

[0052] deforming the original template image according to the original user image and an image deformation algorithm to obtain a target template image corresponding to the original user image;

[0053] inputting the original user image and the target template image into a trained image generation model, processing corresponding regions between the original user image and the target template image according to different processing weights corresponding to facial regions and non-facial regions, and obtaining a target ID photo;

[0054] wherein the image deformation algorithm is obtained by constraint optimization on a thin plate spline (TPS) deformation algorithm, and the constraint includes one or more of a facial key point constraint, a light and shadow consistency constraint, and a facial structure constraint; and the image generation model is obtained by optimization training on a diffusion algorithm.

[0055] As can be seen, in the ID photo generation method provided by the present application, after obtaining an original user image and an original template image (ID photo template) corresponding to the original user image, the original template image is deformed according to the original user image and an image deformation algorithm to obtain a target template image corresponding to the original user image and meeting one or more of a facial key point constraint, a light and shadow consistency constraint, and a facial structure constraint, and then the original user image and the target template image are input into a trained image generation model (obtained by optimization training on a diffusion algorithm) to process corresponding regions according to different processing weights corresponding to facial regions and non-facial regions, so as to ensure that the obtained target ID photo is more natural and real in vision, reliably meets the specification requirements of the actual scene for ID photos, and reduces the calculation time and cost, thereby meeting the large-scale batch generation of ID photos. BRIEF DESCRIPTION OF DRAWINGS

[0056] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings. The same or similar components have the same or similar reference numbers regardless of the drawing number. It should be understood that the drawings are schematic and elements in the drawings are not necessarily to scale.

[0057] Figure 1 A flowchart of a method for generating an ID photo according to an embodiment of the present application;

[0058] Figure 2 A flowchart of a method for generating an ID photo according to an embodiment of the present application;

[0059] Figure 3 A flowchart of a method for generating an ID photo according to an embodiment of the present application;

[0060] Figure 4 A flowchart of a method for generating an ID photo according to an embodiment of the present application;

[0061] Figure 5 A flowchart of a method for generating an ID photo according to an embodiment of the present application;

[0062] Figure 6 A flowchart of a method for generating an ID photo according to an embodiment of the present application;

[0063] Figure 7 A flowchart of a method for generating an ID photo according to an embodiment of the present application; DETAILED DESCRIPTION

[0064] The embodiments of the present application will be described below with reference to the accompanying drawings. The terms used in the embodiments of the present application are only used to explain the embodiments of the present application, and are not intended to limit the present application. The embodiments of the present application will be described below with reference to the accompanying drawings, and those skilled in the art can know that the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems as new scenes appear and technology develops.

[0065] In addition, the terms "first", "second", and the like used throughout the description and drawings of the present application are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, which is only a distinguishing way used in the description of the embodiments of the present application to describe the objects with the same attributes. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or devices containing a series of units do not necessarily limit to those units, but can include other units not clearly listed or inherent to these processes, methods, products or devices.

[0066] With the significant progress of Artificial Intelligence (AI) technology in face recognition and image generation in recent years, compared with the traditional image processing software using Adobe Photoshop (PS), the manual editing and modification operation on the photographed user image to obtain the required ID photo, proposes to use various online ID photo generation tools such as Nais ID Photo Assistant, Simple AI and Picsman (online ID photo making tools), or open source ID photo generation tools such as IDIFY (Hivision ID Photos) or AI-IDPhoto, which will automatically generate a user ID photo matching the ID photo template after uploading the user image and selecting the ID photo template, without the need for manual operation, which is more simple and convenient, cost-saving and improves the efficiency of ID photo generation.

[0067] However, in the process of automatically generating a user ID photo using the above ID photo generation tools, the face region of the user image is usually identified using image segmentation and key point detection technology based on deep learning, the face region key point features are extracted and fused with the ID photo template to obtain an ID photo with the requirements of ID photo template clothing, hairstyle and background. But this way is easy to cause the user face contour and the ID photo template to be not natural, resulting in the generated ID photo not natural enough in detail, and easy to produce unreal visual perception. It may also be due to the ID photo template not meeting the standard ID photo specification requirements, such as uneven lighting, background interference and other factors, resulting in the generated ID photo not meeting the strict requirements of standardized ID photo in various scenarios.

[0068] In order to further improve the above problems, the present application proposes a new ID photo generation method, which will be described in detail below in combination with the drawings.

[0069] Reference Figure 1 The flowchart of the ID photo generation method proposed by the first embodiment of the present application can be applied to an electronic device, which can be a server or a terminal device with data processing capability, such as Figure 1 The ID photo generation method proposed by the present embodiment includes but is not limited to the following steps:

[0070] Step S11, obtaining an original user image and an original template image for the original user image; the original template image comes from an ID photo template library;

[0071] In the embodiments of the present application, the original user image can be at least one image containing the user's head captured by the image collector integrated or externally connected to the electronic device. In the scenarios such as online identity authentication, remote office, online education, enterprise employment process or online certificate handling, the image collector can be started to capture the user's head to obtain one or more clear original user images. During the capturing process, the corresponding capturing prompts can be output according to the requirements of the certificate photo specifications in the corresponding scenario, to prompt the user to adjust the shooting angle, environment or action / pose, etc., so as to obtain high-quality original user images. The implementation method of the present application for online acquisition of original user images is not limited.

[0072] Of course, the present application can also select one or more original user images that can be used to generate a certificate photo from the local storage space of the electronic device or other storage devices, and the source of the original user image is not limited by the present application. In the case of obtaining multiple original user images in the present application, one original user image can be selected by comparison and analysis before the subsequent steps are performed, and the other original user images are not processed. Alternatively, each original user image obtained can participate in the subsequent certificate photo generation step, and the most suitable target certificate photo can be selected from the respective generated certificate photos. The certificate photo generation steps for each original user image are the same, and the present application will not be described one by one.

[0073] In different certificate photo generation scenarios as listed above, the specification requirements for standardized certificate photos are often different, such as background color, user clothing or hairstyle, etc. Therefore, when determining the target type of certificate photo generated from the original user image, such as an ID card, a student card, a visa application or a work certificate, the user can manually select a matching certificate photo template from the standardized certificate photo template library according to the corresponding specification requirements to determine the original template image for the original user image. That is, in response to the selection operation of the output multiple certificate photo templates belonging to the target type, the original template image for the original user image is determined, and the implementation process will not be described in detail.

[0074] In a possible implementation, the electronic device can also use an image search engine to search for a photo template matching a scene where the original user image is located or an input photo condition from a photo template library, and determine the original template image. For example, scene information recognition is performed on a scene where the original user image is collected online, and according to the recognized scene information, an original template image meeting the photo specification requirement under the scene information is selected from a photo template set belonging to a target type that matches the scene information. Alternatively, a photo condition input by a user (such as a mouse, a joystick, or a voice input method) is obtained, and a photo template with the highest matching degree to the photo condition is searched from the photo template library, and is used as the original template image. The implementation method of the original template image is not limited in the present application, and can be determined as needed.

[0075] In step S12, the original template image is deformed according to the original user image and an image deformation algorithm to obtain a target template image for the original user image. The image deformation algorithm is obtained by constraint optimization on a thin plate spline (TPS) deformation algorithm, and the constraint includes one or more of a face key point constraint, a light and shadow consistency constraint, and a face structure constraint.

[0076] As described above in relation to the original template image, the original template image is directly selected from the photo template library, and the photo features presented are fixed and single, which can have a low matching degree with the specification requirement of a standard photo under an actual scene, so that the generated photo cannot meet the strict requirement of the standard photo under the actual scene, and a lot of time and effort are needed to edit and modify the generated photo.

[0077] To this end, the present application proposes to dynamically adjust the original template image, to realize high-precision deformation of a face key region (such as a region where an eye, a nose tip, a mouth corner, a chin, or the like is located) on the basis of preserving a light and shadow effect, so that the deformed face key region meets the specification requirement of a standard photo under an actual scene, and a non-face key region can be flexibly and smoothly transitioned to preserve the features of the non-face key region. According to need, the face region structure can also be adjusted to be left-right symmetrical, to ensure that the face of the user in the finally generated target photo is natural.

[0078] In the embodiments of the present application, a modified TPS (Thin Plate Spline) deformation algorithm can be used to perform affine transformation on the original template image according to the facial key point features in the original user image, such as one or more transformation operations of overall scaling, translation and rotation of the face region in the original template image, to quickly match the approximate shape and position of the face in the original user image. In this way, personalized matching of different face shape features (such as round face, square face or long face, etc.) is realized, and in combination with other constraints, fine deformation of the local (i.e. the facial key region) of the original template image can also be performed to solve the detail matching and contour distortion problem, so that the deformed template image perfectly fits the face shape in the original user image, while retaining the skin texture and light and shadow effects in the original template image, and also constraining the left-right symmetry of the face in the deformed template image to make the visual effect consistent.

[0079] It can be seen that for different original user images, even if the same original template image is selected, the target template image obtained by the image deformation method described above can also be different, so that it not only meets the standard requirements of the standardized ID photo, but also realizes high-quality, natural fitting and personalized visual presentation, and is used to realize the subsequent ID photo generation step. Compared with directly using a fixed original template image, the present application improves the diversity of ID photo templates by this dynamic adjustment of the template image processing method, meets the personalized needs, can adapt to various lighting, background and resolution conditions, ensures that the generated ID photo meets the application standards of various scenes, and greatly improves the user experience and satisfaction.

[0080] It should be noted that the constraints of the image deformation algorithm used in the present application can be flexibly adjusted or modified according to actual scene requirements, and are not limited to the several constraints listed above. In combination with the above analysis, the facial key point constraint is used to indicate the dynamic weighting mechanism for the facial key region and the non-facial key region, so as to adopt different deformation weights to participate in the image deformation processing. The light and shadow consistency constraint is used to make the light and shadow effects of the template image before and after deformation consistent. The face structure constraint is used to indicate the left-right structure symmetry of the face in the deformed template image. The present application does not limit the method of how to realize these constraints.

[0081] In step S13, the original user image and the target template image are input into the trained image generation model, and the corresponding regions between the original user image and the target template image are processed according to different processing weights corresponding to the facial region and the non-facial region, to obtain a target ID photo; the image generation model is obtained by optimizing and training the diffusion algorithm.

[0082] After the above analysis, the target template image fully meets the standard requirements of the standardized ID photo in the actual scene, solving the problems of distortion of the facial key area, uncoordinated light and shadow, and facial asymmetry in the original template image. The target template image and the original user image are input into the image generation model for processing to ensure that the output target ID photo meets the standard requirements of the standardized ID photo.

[0083] Since the image generation model of the present application is obtained by optimizing training through the diffusion algorithm, it can be an optimized diffusion model. Regardless of the forward noise adding process or the reverse noise removing process, different processing weights are configured for the facial area and the non-facial area of the image, so that the image generation model processes the two areas differently, preserves the user's facial features and template light and texture, and ensures the generation of natural and high-quality professional target ID photos. The present application does not describe the implementation process of forward noise adding and reverse noise removing in detail.

[0084] In summary, in the automatic generation of user ID photos, after selecting an ID photo template that meets the requirements of the ID photo scene, i.e. the original template image, the image deformation algorithm is used to deform the image to obtain a target template image that meets the standard ID photo requirements in the current ID photo scene, i.e. dynamically adjusting the ID photo template image. Since the image deformation algorithm is optimized based on one or more of the TPS deformation algorithm, such as facial key point constraint, light consistency constraint, and facial structure constraint, the adjusted target template image and the original user image are input into the image generation model, i.e. the optimized diffusion model, to reliably and quickly generate a target ID photo that meets the standard ID photo requirements, so that it can accurately preserve the skin texture, light effect, and background details of the target template image, while maintaining the similarity of the user's facial features, making the target ID photo more natural and realistic in vision, and reliably meeting the standard requirements of the actual scene for ID photos. Through the denoising and restoration process of the diffusion model, the calculation time is effectively shortened and the calculation resource consumption is reduced, so that the present application can be applied to ordinary devices with limited resources and meet the demand for large-scale batch generation of ID photos.

[0085] Reference Figure 2 The flowchart of the ID photo generation method of Embodiment Two of the present application is described. This embodiment describes a selectable implementation process for how to deform the original template image to obtain a target template image in the above-mentioned ID photo generation method, as shown in Figure 2 The implementation process can include but is not limited to the following steps:

[0086] Step S21, detecting the facial key points of the original template image to obtain a plurality of original facial key point positions, and detecting the facial key points of the original user image to obtain a plurality of user facial key point positions;

[0087] Step S22, dynamically matching the user face key point position to obtain a plurality of target face key point positions;

[0088] In combination with the foregoing description of the technical solutions of the present application, in order to reduce the face matching error, the present application determines the accurate matching of the user facial feature in the original user image and the face key point of the ID photo template image through high-precision facial feature point positioning and geometric transformation, avoids the deviation of the face contour and unnatural phenomena, and improves the accuracy and authenticity of the subsequent generated ID photo. For this purpose, the present application can detect the face key points of the original template image and the original user image respectively to determine the respective face key point positions, detect the user face key points through dynamic matching, generate target face key point positions, and implement subsequent matching deformation processing.

[0089] In some embodiments, the present application can use a constrained local model (CLM) or an active shape model (ASM) or an active appearance model (AAM) to locate the face key points. Other key point detection models trained by deep learning methods can also be used to identify the face key points in the image, and the present application does not limit the face key point detection method. For convenience of description, the original template image can be denoted as I m The coordinates of each original face key point position detected therefrom can be represented as (x i , y i ), and therefore, the original face key point set P m formed by all original face key points can be represented as: P m = { (x i , y i )}.

[0090] According to the foregoing face key point detection method, on the basis of determining the user face key points in the original user image, the user face key points can be adjusted to the key points of the target pose or shape / expression, i.e., the target face key points, through dynamic matching or optimization algorithm. The target pose or shape / expression can be determined according to the actual scene, and then the user face key point position can be adjusted through the dynamic matching method based on interpolation, optimization or deep learning to obtain a plurality of target face key point positions, and the set formed thereby, the target face key point set P may be represented as: Preferably, the present application can maintain the high-resolution feature map of the input image (original user image) through a high-resolution network (HRNet), improve the key point positioning accuracy, support multi-face detection, have robustness to occlusion and extreme poses, and reduce the shooting requirements for the original user image.

[0091] wherein the interpolation-based dynamic matching is to gradually adjust the user facial key point positions to the target facial key point positions through an interpolation method (such as linear interpolation or spline interpolation); the optimization-based dynamic matching is to minimize the position difference between the user facial key points and the target facial key points through an optimization algorithm (such as gradient descent); and the deep learning-based dynamic matching is to learn the mapping relationship from the user facial key points to the target facial key points using a deep learning model (such as a generative adversarial network or a transformer), so as to map the actually detected user facial key point positions to the target facial key point positions using the trained model, and the implementation process is not described herein.

[0092] In addition, in actual applications, the number of the detected facial key points can be 98 key points, 68 key points, 29 key points, 21 key points, etc., which can be determined according to actual needs. In the present application, the number of the original facial key point positions can be the same as the number of the target facial key point positions, i.e., P m and The number of elements in the two sets is the same.

[0093] In step S23, the facial key regions of the original template image are non-linearly deformed through an image deformation algorithm according to the plurality of original facial key point positions and the plurality of target facial key point positions, and the facial non-key regions of the original template image are smoothly transitioned, and a deformed template image satisfying the facial key point constraint is obtained through bilinear interpolation calculation.

[0094] In the embodiments of the present application, the target deformation function for the original template image can be obtained by minimizing the image deformation algorithm according to the plurality of original facial key point positions and the plurality of target facial key point positions, so as to map each pixel point (x, y) in the original template image I m to the corresponding target position through the target deformation function, and a deformed template image I is obtained through bilinear interpolation calculation. In the solving process of the image deformation algorithm, the distance between the original facial key point position and the target facial key point position corresponding to the same facial key point can be minimized, and the deformed template image can be optimized in combination with the following constraints to obtain a target template image that conforms to the standard requirements of the standardized ID photo in the actual scene. The solving process of the improved TPS deformation algorithm is not described herein.

[0095] In a possible implementation, in the determination of P m and Afterwards, the TPS morphing algorithm can be applied to map each original facial key point position in P m to the corresponding facial key target facial key point position in P , to realize the nonlinear flexible deformation of the facial key region (i.e., the region belonging to the predefined facial key points, such as the eyes, the tip of the nose, the corners of the mouth, the cheeks, the forehead, the lower jaw, and the chin) in the original template image, and to realize the smooth transition of the non-key facial region (such as the outer edge of the face and the background transition region) in the original template image, thereby improving the facial shape fitting degree. As analyzed above, the present application can map each pixel point (x, y) in the original template image to the new coordinate, i.e., the target position , through the deformation function in the image morphing algorithm, but there is a non-integer coordinate point in the target position, and the pixel value corresponding to the template image after the above mapping and deformation processing can be calculated by using the integer coordinate points around the target position through the bilinear interpolation method, denoted as , and the calculation process is not described in detail.

[0096] In step S24, the deformed template image is processed according to at least one of the light and shadow consistency constraint and the facial structure constraint, to obtain the target template image for the original user image.

[0097] In the deformation process of the original template image using the image morphing algorithm, in order to ensure the natural transition of the light and shadow effect and the skin texture, the light and shadow effect difference before and after the image deformation can be added to the deformed template image according to the light and shadow consistency constraint in the image morphing algorithm, so that the deformed template image has the light and shadow effect of the original template image, i.e., the coordination and natural transition of the light and shadow of the deformed template image are ensured. Optionally, the light and shadow effect difference can be calculated according to the parameters in the template image that affect the light and shadow effect, such as the light and shadow intensity difference before and after the deformation, and the calculation process is not described in detail.

[0098] Preferably, in order to further improve the overall visual effect of the deformed template image and improve the visual effect of the generated ID photo, in the deformation process of the original template image using the image morphing algorithm, the mapping of the left and right facial key point positions in the template image can also be smoothly constrained according to the facial structure constraint, such as the facial symmetry constraint, in the image morphing algorithm, to ensure the left and right structure symmetry of the facial region in the generated image.

[0099] Based on this, in some embodiments, a face alignment method based on affine transformation of face key points can be adopted, and symmetric key points can be generated based on a deep learning model. In addition, on the basis of face alignment, symmetry adjustment, symmetry detection and correction, and other methods can be used to achieve the same. In addition, the pre-defined symmetry template or dynamic symmetry adjustment can be used to achieve the same. The present application does not limit the method for achieving the symmetry constraint deformation of the face region in the deformation process of the original template image.

[0100] Referring to Figure 3 The flowchart of the method for generating an ID photo according to the third embodiment of the present application is shown in FIG. 3. This embodiment describes another optional method for deforming the original template image to obtain the target template image in the method for generating an ID photo described above. As shown in FIG. 3, the implementation process can include but is not limited to the following steps: Figure 3

[0101] In step S31, a plurality of original face key point positions of the original template image and a plurality of corresponding target face key point positions are obtained.

[0102] The implementation method of step S31 can refer to the description of the corresponding part of the above embodiment, which will not be described here.

[0103] In step S32, the deformation weight corresponding to each face key point and face non-key point is obtained. The deformation weight corresponding to the face key point is greater than the deformation weight corresponding to the face non-key point, and the deformation weight corresponding to the face key point with a higher impact on the quality of the ID photo is greater.

[0104] In the embodiment of the present application, according to the different importance (i.e. impact) of different position points of the face region on generating a high-quality ID photo, the face region is divided into a face key region and a face non-key region. The face key region can be further divided into a double-eye region, a nose tip region, a mouth corner region, a cheek region, a forehead region, and a chin and jaw region, etc. At the same time, according to the different importance, different deformation weights can be configured for each face region to represent the degree of deformation of the corresponding face region in the deformation process of the original template image, so as to avoid deformation distortion. The correspondence between different importance and different deformation weights of different face regions can be determined according to experience or historical data, or other dynamic weight allocation methods can be used, which are not limited in the present application.

[0105] In one possible implementation, according to the face regions divided in the above example, the pre-configured deformation weight w i As shown in Figure 4 ​The table content shown, need to explain, for each face region corresponding to the deformation weight value includes but not limited to Figure 4 The size shown, and the division type of the face region includes but not limited to Figure 4 The content shown, can be adjusted according to the actual scene needs, this application does not make detailed description. Based on this, in the actual application of the embodiment, the table mode shown can be used, or the position range of each face region is determined, and the segmented function between each position range and the deformation weight is generated and stored, so as to be directly read. Of course, this application can also dynamically allocate the corresponding deformation weight to each face region according to the pre-learned weight distribution rule, and the implementation process is not described in detail. Figure 4 The table mode shown, or the position range of each face region is determined, and the segmented function between each position range and the deformation weight is generated and stored, so as to be directly read. Of course, this application can also dynamically allocate the corresponding deformation weight to each face region according to the pre-learned weight distribution rule, and the implementation process is not described in detail.

[0106] Step S33, according to different deformation weights, by mapping a plurality of original face key point positions to the target face key point positions of the corresponding face key points, realizing the nonlinear deformation of the face key region and the smooth transition of the face non-key region of the original template image, so as to minimize the distance between the original face key point position and the target face key point position corresponding to the same face key point;

[0107] In the image deformation algorithm proposed in the application, that is, the improved TPS deformation algorithm, different deformation weights can be applied to the face key region and the face non-key region through the dynamic region weighting mechanism, to ensure high-precision matching of the face key region in the template image deformation process, and to avoid distortion by smooth transition of the face non-key region. Among them, since different face key regions have different influence on the quality of the ID photo, different deformation weights are configured, such as Figure 4 As shown, further improve the high-precision deformation of different face key regions, and ensure the natural fit of the face.

[0108] Based on the above analysis, when the original template image is subjected to affine transformation by using the deformation function (i.e. interpolation function) obtained by solving the image deformation algorithm, in the implementation process, in the face key point constraint term of the bending energy function, in order to ensure that the deformed face key point is as close as possible to the corresponding target key point, a face dynamic weighting mechanism is used to give different deformation weights w i , which can be expressed as:

[0109] ; (1)

[0110] Wherein, represents the target position (new coordinates) of the i-th target face key point, represents the deformed position of the i-th original facial key point. i=1, 2, …, n, n represents the number of facial key points, such as 98, etc. The value of n is not limited in the present application, and can be determined as appropriate. As can be seen, in the process of mapping each pixel point in the original template image to the new coordinates, i.e., the target position, the deformation weight w i different, so that the deformation degree of the corresponding position is different, thereby better meeting the pixel position deformation requirement of the facial region at different positions, minimizing the distance between the original facial key point position and the target facial key point position corresponding to the same facial key point, ensuring high-precision matching of the facial key region, and smooth transition of the facial non-key region, avoiding deformation distortion.

[0111] It should be understood that, in combination with the operation principle of the TPS algorithm, in the process of deforming the original template image, the curvature of the deformation function can be constrained by the second-order partial derivative, so that the deformation is smooth and natural. Therefore, the bending energy function of the image deformation algorithm also includes the following smooth regularization term:

[0112] ; (2)

[0113] where λ can represent a smooth regularization parameter for adjusting the constraint strength, and its value can be determined according to actual requirements. The value of λ and its size are not limited in the present application.

[0114] In combination with the above analysis, in the process of deforming the original template image, the pixel value of the pixel point in the deformed template image obtained after the facial key point constraint and the smooth regularization processing can be represented as: As analyzed above, the pixel value can be calculated by the bilinear interpolation method:

[0115] ; (3)

[0116] In formula (3), i=0, 1, …, m, m represents the width pixel value of the original template image; j=0, 1, …, n, n represents the height pixel value of the image, for example, the size of the original template image is 640×720, m=640, n=720. represents the original pixel value of the i-th row and j-th column pixel point in the original template image, w ij The interpolation weight of the i-th row and j-th column pixel point can be determined by the distance between the original coordinates and the target coordinates of the pixel point. Optionally, in the bilinear interpolation, the interpolation weight and the distance are inversely proportional to each other. The closer (i.e., the smaller the distance) the original coordinates of the pixel point in the original template image to the target coordinates, the greater the interpolation weight. The interpolation weight can be calculated in the following manner, but is not limited to the following manner:

[0117] ; (4)

[0118] wherein (x i , y i ) represents the original coordinates (x, y) of the pixel point are mapped to the target position After that, the non-integer coordinates are adjusted to integer coordinates.

[0119] Step S34, the first light and shadow intensity of the original template image and the second light and shadow intensity of the deformed template image are obtained;

[0120] Step S35, the light and shadow residual between the original template image and the deformed template image is obtained according to the difference between the first light and shadow intensity and the second light and shadow intensity of the same pixel point;

[0121] Step S36, the deformed template image is smoothed according to the light and shadow residual and the light and shadow constraint weight, so that the light and shadow effect of the processed template image is consistent with the light and shadow effect of the original template image;

[0122] As analyzed above, in the deforming process of the original template image, in addition to considering the face key point constraint, the light and shadow consistency constraint is also considered, that is, the light and shadow effects of the template image before and after deformation are ensured to be consistent, so that the light and shadow in the deformed template image is consistent with the skin texture. In a possible implementation, the light and shadow effect of the template image can be maintained by calculating the light and shadow residual.

[0123] Based on this, the light and shadow consistency constraint R light can be obtained according to the following formula:

[0124] ; (5)

[0125] In formula (5), P i represents the position (coordinate value) of the i-th original face key point in the original template image, and for this , the light and shadow intensity of the position can be represented, which is recorded as the first light and shadow intensity, , the target position of the i-th target face key point in the deformed template image can be represented, and for this , the light and shadow intensity of the target position can be represented, which is recorded as the second light and shadow intensity. The difference between the first light and shadow intensity and the second light and shadow intensity corresponding to the same face key point is calculated, the difference between the original template light and shadow and the target template light and shadow of the corresponding face key point is obtained, that is, the light and shadow intensity difference, and the square sum of the light and shadow intensity difference of each pixel point is taken as the light and shadow consistency constraint term R light of the bending energy function in the image deformation algorithm, so as to realize the calculation of the deformation function.

[0126] In addition, for R light A corresponding control weight, i.e., a light and shadow constraint weight, can also be configured to control the importance of the light and shadow consistency constraint, which can be denoted as γ, and the value range can be 0.01-0.1, etc., which can be flexibly configured or adjusted according to the importance requirement of the light and shadow consistency constraint in the actual scene. Based on this, the light and shadow residual of each pixel point described above can be added to the deformed template image, such as adding the light and shadow residual of each pixel point to the pixel value of the corresponding pixel point, so that the light and shadow effects of the template image before and after deformation are consistent.

[0127] Step S37, determining the face symmetry line in the deformed template image;

[0128] Step S38, performing point mirror transformation on the deformed template image according to the face symmetry line and the symmetry constraint weight, so that the face key points in the deformed template image are left-right symmetric about the face symmetry line.

[0129] In the deformation process of the original template image, the face region of the original template image can also be optimized according to the face structure constraint. In combination with the related description of the face structure constraint in the foregoing, the key points of the face region need to be left-right symmetric to ensure that the face structure is left-right symmetric. The face structure constraint R symmetry of the present application can be realized by using the following formula:

[0130] (6)

[0131] In formula (6), Sym() can represent a symmetric transformation operation such as point mirror transformation, to ensure that the deformed face key points are symmetric about the face symmetry line (such as the face center axis). P left represents the key point set of the left face region (left face), and P right represents the face key point set of the right face region (right face). It should be understood that the positions of P left and P right in the expression formula (6) of the face structure constraint can be exchanged. The pixel / key point in the present application can be represented by its coordinate value (position) in the image, so the key point set in the present application refers to the key point position set, which will not be explained one by one in the following.

[0132] In actual application of the present embodiment, the center points of the face key region can be used to construct the face symmetry line, i.e., the face center axis. For example, the center points of the left eye, the right eye, the nose, the forehead, the lower lip, and the chin can be selected as key point positions, and then the horizontal center axes can be calculated, such as calculating the average value C x, the position of the vertical pair of vehicle axles is determined, and then the vertical center line equation representing the vertical center axis is determined, i.e. x = C x , which is a straight line perpendicular to the x-axis. Based on this, for any face key point P(x, y) in the original template image, the symmetric point x of the symmetric point of the center axis (x = C x ) can be calculated. , , =2 C x -x.

[0133] In this way, the embodiment can realize the point mirror transformation of the face region in the deformed template image in this way, and determine the pixel value of the corresponding pixel point after transformation. As analyzed above, based on the face key point constraint, the pixel value of each pixel point in the original template image after mapping is calculated by bilinear interpolation and is denoted as , the pixel value of the corresponding pixel point can be denoted as after the light and shadow residual error is introduced into the deformed template image by the above method for compensation and optimization, and the face symmetry optimization in the deformed template image can be updated according to the following formula:

[0134] ; (7)

[0135] In formula (7), the content on the left side of the equal sign represents the pixel value of the pixel point in the finally generated target template image, and the numerator on the right side of the equal sign represents the pixel value after the light and shadow consistency optimization and the face structure symmetry optimization, respectively. The average value is taken as the final pixel value. As can be seen, the present application

[0136] In this way, the visual smoothness of the target template image is included, the light and texture are naturally transitioned, and the face region is left-right symmetric, which facilitates subsequent obtaining of high-precision face fitting and preserves the light and shadow effects and skin texture features of the template image.

[0137] In the above point mirror transformation process, the symmetry constraint weight δ of the face region can be combined to realize it. The δ can usually take a value of 0.05-0.2, and the specific value size is not limited and can be determined as needed. It should be noted that the way to realize the left-right symmetry of the face structure includes but is not limited to the calculation method listed above.

[0138] Based on the above analysis, the bending energy function of the image deformation algorithm proposed in the present application, i.e. the bending energy function of the improved TPS deformation algorithm, can be represented as:

[0139] ; (8)

[0140] ; (9)

[0141] From formula (9), compared with the bending energy function of the traditional TPS deformation algorithm, the application optimizes it by face key point constraint, and introduces deformation weight w i for different face regions to ensure high-precision deformation of the key face region and flexible and smooth transition of the non-key face region. Compared with the same deformation degree processing of each face region, the dynamic region weighting mechanism of the application can effectively solve the detail matching and contour distortion problems. As shown in formula (8), in order to ensure that the deformed template image perfectly fits the original user image face shape while retaining the matching texture and light and shadow effect of the template image, the application also introduces the light and shadow consistency constraint R light and the face structure constraint R symmetry .

[0142] In actual application, the bending energy function E final (f) is minimized to obtain the deformation function (i.e. interpolation function) f, which can be expressed as f(x, y). Since the face key point constraint, smooth regularization, light and shadow consistency constraint and face structure constraint are considered in the solving process, each pixel point in the original template image is mapped to the target position using the deformation function f(x, y), and the deformed template image is obtained. According to the mapping relationship obtained, the deformed pixel points are subjected to interpolation transformation, which ensures that the face region of the generated target template image fits the user face shape, solves the detail matching and contour distortion problems, and the light and shadow and skin texture are naturally transitioned, the face region is left-right symmetrical, and the overall visual effect is improved.

[0143] Referring to Figure 5 , a flowchart of the method for generating an ID photo according to the fourth embodiment of the application is shown, and the embodiment can further describe the optional implementation process of how to obtain the target ID photo by inputting the original user image and the target template image into the trained image generation model in the above method for generating an ID photo. As shown in Figure 5 , the detailed implementation method includes the following steps.

[0144] In step S51, the original user image and the target template image are input into the trained image fusion model, the face region of the original user image is subjected to feature extraction to obtain a first feature vector, and the target template image is subjected to feature extraction to obtain a second feature vector.

[0145] The original user image I p and the target template image After the trained image fusion model is input, in order to control the fusion ratio of the original user image and the target template image, the personalization and standardization of the generated ID photo are dynamically adjusted. In this regard, the application can combine the feature vectors of the input images to calculate a dynamic weighting coefficient as the fusion weight of the two input images in the forward noise adding process and the restoration weight of the two images in the reverse denoising process.

[0146] In order to obtain a suitable dynamic weighting coefficient, feature extraction can be performed on the input images through a pre-trained feature extraction network (such as an InsightFace-based feature extraction model) to obtain corresponding feature vectors. Among them, for the feature extraction of the original user image, the feature extraction is performed on the face region, that is, the features of the face key region in the original user image are extracted, that is, the positions and shapes of the face key regions such as eyes, nose, and mouth, etc. The feature vector extracted therefrom is denoted as the first feature vector F user . And the target template image can be directly extracted to obtain the second feature vector F template . The implementation process is not described in detail.

[0147] Step S52, similarity calculation is performed on the first feature vector and the second feature vector to obtain an image similarity;

[0148] Step S53, the face key point positions between the original user image and the target template image are matched to obtain a face key point matching degree;

[0149] In the embodiment of the application, similarity calculation methods such as cosine similarity algorithm or Euclidean distance can be used to realize the similarity calculation between the first feature vector and the second feature vector to obtain the image similarity, which can be denoted as Sim(F user , F template ). This similarity calculation method or other key point matching algorithm can also be used to realize step S53. Taking the Match() key point matching algorithm as an example, the face center points of the original user image and the target template image can be aligned, and then the relative average offset between the aligned original user image and the target template image is calculated to obtain the face key point matching degree between the original user image and the target template image, which can be denoted as Match(P user , P template ).

[0150] Step S54, the image similarity and the face key point matching degree are weighted and summed, and after normalization processing, a dynamic weighting coefficient for the original user image and the target template image is obtained;

[0151] According to the above analysis, the dynamic weighting coefficient λ can be calculated by the following formula:

[0152] ; (10)

[0153] In this embodiment, the Sigmoid function can be used to achieve normalization. For the image similarity and facial key point matching degree obtained above, as shown in formula (10), the corresponding weighting coefficients α and β can be configured to control the importance of image similarity and facial key point matching degree. This application does not limit the value of these two weighting coefficients, and they can be configured or adjusted based on experience or historical data.

[0154] Step S55: Based on the dynamic weighting coefficients and dynamic face mask, the facial and non-facial regions of the original user image and the target template image are fused and noise-added respectively to obtain a fused and noise-added image.

[0155] In this embodiment of the application, a dynamic facial mask M can be constructed. face Used to distinguish facial and non-facial regions in an image, such as M face The regions with an element value of 1 represent facial regions, and the regions with an element value of 0 represent non-facial regions. Thus, the facial and non-facial regions of an image can be determined by multiplying it with the corresponding image.

[0156] The noise addition operation in step S55 can be implemented using a normal distribution algorithm, i.e. This allows us to base our analysis on the mean μ and variance. Generate a noisy image. Wherein, variance... The noise intensity at time step t represents the noise intensity that increases with time step t during the noise addition process, controlling the randomness of the noise addition process. This application does not detail the implementation process of this noise addition method. Thus, the simulation of the gradual degradation process of an image from high quality to pure noise yields a noisy intermediate image, i.e., the fused noisy image I. t :

[0157] ; (11)

[0158] Based on this, in one possible implementation, the original user image and the target template image can be fused according to dynamic weighting coefficients to obtain a fused image I. mix :

[0159] ;(12)

[0160] Subsequently, based on the dynamic facial mask, noise can be added to the facial region of the fused image, and noise can be gradually added to the non-facial region of the target template image. The noisy facial region and the noisy non-facial region are then fused to obtain the fused noisy image I. t :

[0161] (13)

[0162] Therefore, in the noise adding process, the face region of the original user image can be gradually added with noise, and the non-face region of the target template image can be added with noise, and then the image regions after noise addition of the two are spliced and fused to obtain a fused noise-added image, and the implementation process is not described herein.

[0163] In step S56, the fused noise-added image is restored according to the dynamic weighting coefficient and the dynamic face mask to obtain the target ID photo.

[0164] In combination with the operation principle of the diffusion algorithm, in the reverse denoising process, the denoising network in the image generation model can be input with the fused noise-added image, the first feature vector and the second feature vector. In one possible implementation, in combination with the implementation method of the forward noise addition process described above, the face region and the non-face region in the fused noise-added image can be gradually denoised according to the first feature vector, the second feature vector and the dynamic weighting coefficient, respectively, to obtain the target ID photo. In the denoising process at each time step t, the feature-guided model can be combined for denoising. In the feature-guided noise removal process, the face region features of the original user image and the target template image can be introduced as conditional input to guide the noise removal in the following manner:

[0165] (14)

[0166] In the denoising process, the denoised image obtained at the last time step can be represented as:

[0167] (15)

[0168] Thus, the restoration weights of the face region and the non-face region of the denoised image can be dynamically adjusted to ensure that the face region features and the light and shadow details can be reliably and completely preserved. After gradual denoising, the final target ID photo is generated at time step t=0. Regarding the different denoising processes of the face region and the non-face region, the M face implementation can be used. Thus, formula (15) can be updated as:

[0169] (16)

[0170] In combination with the processing process of the image generation model on the input image described above, in the training process of the image generation model, the following methods can be implemented, but are not limited thereto:

[0171] By obtaining real user images in multiple lighting conditions, different face shapes, different expressions and skin colors as sample user images, the adaptability of the image generation model to diverse inputs can be ensured. Similarly, standardized ID photo images with different lighting effects and background styles can be obtained as initial sample template images. Then, the sample user images and the initial sample template images can be preprocessed respectively to realize subsequent model training.

[0172] In the preprocessing process of the sample user images, geometric enhancement operations such as random rotation, scaling or cropping can be used to enhance the robustness of the model to different poses, and color attribute information such as brightness, contrast and chroma can be adjusted to simulate the light changes in real shooting. Pseudo-objects and light changes can also be added to improve the model's learning ability for light characteristics. The present application does not limit the preprocessing method. The adjustment process of the initial sample template image is similar to the adjustment process of the original template image described in the above embodiment, and will not be described here.

[0173] Then, a pre-trained model such as a feature extraction network can be used to extract a first sample feature vector of the sample user image and a second sample feature vector of the sample template image, and the feature consistency and face shape matching can be realized by combining the face key point positions. After obtaining these contents, the initial image generation model can be used to process the sample user image and the sample template image according to the method described above to obtain a predicted ID photo. Then, a variety of loss functions can be used to calculate the feature consistency loss, the light consistency loss, the adversarial loss and the reconstruction loss respectively to optimize the quality of the generated predicted ID photo, and to ensure the preservation of the user's facial features, the naturalness of the details and the overall visual effect of the template light.

[0174] Optionally, the feature consistency loss L feature :

[0175] ; (17)

[0176] where F generated may represent the sample feature vector of the predicted ID photo, and F may represent the high-dimensional feature vector of the predicted ID photo in the face region. The dynamic weighting coefficient λ calculation method is similar to the method described above and will not be described in detail. By minimizing the feature consistency loss, F generated can be as close as possible to the weighted fusion result of the sample feature vector of the sample user image and the sample feature vector of the sample template image.

[0177] Optionally, the application can obtain the light and shadow consistency loss L according to the sample light and shadow intensity of the sample fused noise image and the sample template image by the following formula light ;

[0178] ; (18)

[0179] In formula (18), L generated can represent the light and shadow residual error of the generated predicted ID photo, i.e., the light and shadow distribution feature, which can be extracted by image gradient analysis or a dedicated light and shadow feature extraction model to measure the brightness, shadow and texture of the image. The implementation process is not described in detail. At the same time, by using this extraction method, the sample light and shadow residual error L template of the sample template image can be extracted as a reference for the standard light and shadow effect, and the implementation process is not described in detail. In the training process, the application hopes that L generated is as close as possible to L template to keep the light and shadow effect of the generated predicted ID photo consistent with the sample template image.

[0180] Optionally, according to the discriminator network, the adversarial loss between the predicted ID photo output by the initial image generation model and the standard ID photo is obtained, which is calculated in the following formula:

[0181] ; (19)

[0182] Where D can represent the discriminator network, which is an adversarial training module for distinguishing between the real standard ID photo I target and the generated predicted ID photo I final , compares and analyzes the input standard ID photo and predicted ID photo, and outputs the probability that the predicted ID photo is a real standard ID photo, i.e., the prediction score. Where I target can be a reference image labeled by artificial labeling or generated with high quality, representing the expected quality of the target generated ID photo, and the operation principle of the discriminator is not described in detail. In the training process, the probability can be maximized to improve the realism of the predicted ID photo.

[0183] In addition, the application can also obtain the reconstruction loss L recon of the predicted ID photo according to the predicted ID photo output by the initial image generation model and the standard ID photo, and then obtain the total predicted loss L total according to the feature consistency loss, the light and shadow consistency loss, the adversarial loss and the reconstruction loss, and the loss weights:

[0184] ; (20)

[0185] Where the above loss weights , β and The initial value can be determined according to experience or historical data, =1, β=0.5 and =0.1, to control the importance of feature consistency, light and shadow consistency and adversarial training, to train the initial image generation model by minimizing the predicted total loss, that is, to adjust the parameters of the initial image generation model, such as the parameters of the diffusion model and the parameters of the discriminator network, to perform iterative training according to a preset learning rate (such as 10 -4 ) until the predicted total loss converges or reaches a preset number of iterations, to obtain an image generation model. The application does not detail the implementation process of iterative training. The image generation model obtained according to the training method proposed in the application can reliably meet the diversified user needs, improve the universality, and ensure that it is strictly in accordance with the standard requirements of the generated personalized ID photo.

[0186] The above describes a method for generating an ID photo provided by an embodiment of the application. The following will introduce a device for executing the above-mentioned ID photo generation method.

[0187] Referring to Figure 6 , a structural schematic diagram of an ID photo generation device according to an embodiment of the application is shown in FIG. 6. The ID photo generation device can include: Figure 6

[0188] An image acquisition module 61 is configured to acquire an original user image and an original template image corresponding to the original user image. The original template image is from an ID photo template library.

[0189] A deformation processing module 62 is configured to perform image deformation on the original template image according to the original user image and an image deformation algorithm, to obtain a target template image corresponding to the original user image.

[0190] A generation module 63 is configured to input the original user image and the target template image into a trained image generation model, to process corresponding regions between the original user image and the target template image according to different processing weights of the face region and the non-face region, to obtain a target ID photo.

[0191] The image deformation algorithm is obtained by constraint optimization on a thin plate spline (TPS) deformation algorithm. The constraint includes one or more of a face key point constraint, a light and shadow consistency constraint, and a face structure constraint. The image generation model is obtained by optimizing and training a diffusion algorithm.

[0192] Those skilled in the art can understand that the functions and technical effects of each module in the above device embodiment are equivalent to the corresponding steps described in the foregoing method embodiments. The specific implementation details can be referred to the description in the method part, which will not be described here.​

[0193] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the ID photo generation methods provided in this application.

[0194] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the ID photo generation methods provided in this application.

[0195] Reference Figure 7 This is a schematic diagram of the hardware structure of an electronic device proposed in an embodiment of this application, as shown below. Figure 7 As shown, the electronic device may include: at least one communication element 71, at least one memory 72, and at least one processor 73, wherein:

[0196] At least one communication element 71, at least one memory 72, and at least one processor 73 can communicate via a bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be categorized as an address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by only two lines, but this does not mean that there is only one bus or one type of bus.

[0197] The communication element 71 may include communication elements supporting wireless communication methods such as Wi-Fi, Bluetooth, and 5G / 6G mobile communication, enabling the electronic device to transmit data with other devices. It may also include one or more interfaces supporting wired communication methods, such as general-purpose input / output (GPIO) interfaces, USB interfaces, and universal asynchronous receiver / transmitter (UART) interfaces, to achieve data transmission between various components within the electronic device. This application does not limit the composition structure of the communication element to achieve this function or its corresponding communication transmission mechanism; it can be determined as appropriate.

[0198] The memory 72 can be configured to store a plurality of computer program instructions for implementing the method for generating an ID photo according to the embodiments of the present application. The processor 73 can load and execute the computer program instructions stored in the memory 72 to implement each step of the method for generating an ID photo according to the embodiments of the present application. The implementation process can refer to the description of the corresponding part of the method embodiments above.

[0199] In the embodiments of the present application, the memory 72 can include a storage medium such as a floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. The processor 73 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA).

[0200] According to actual needs, the electronic device can further include at least one communication element for receiving the original time sequence data sent by the acquisition device or sending to other devices (such as servers, etc.) for processing, and can also be used for data or instruction transmission between internal components of the electronic device.

[0201] It should be understood that, Figure 7 It should be understood that, Figure 7 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of the present application. In actual applications, the electronic device can include more or fewer components than those shown, or some components can be combined. In the case of a terminal device such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a smart transportation / medical device, a smart home device or a robot, it can also include, for example, a sound pickup device, a speaker, a display screen, various sensors, an antenna, a power module, a radio frequency component, an external port and other input / output components, etc. The processing function requirements can be determined, and the present application will not be described one by one.

[0202] It should be noted that the apparatus embodiments described above are merely exemplary, and the units described as separate units can or can not be physically separate, and the units displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. In addition, the connection relationship between the modules in the apparatus embodiment provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0203] Through the description of the above embodiments, in the above embodiments, all or part of the embodiments can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of the computer program product can be realized. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that the computer can store or be integrated into a training device, data center, etc. Data storage device containing one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0204] In addition, each embodiment in the specification is described in a progressive or parallel manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between each embodiment can be referred to. For the apparatus, electronic device, product and medium disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the method part.

Claims

1. A method of generating an ID photo, characterized by, The method comprises: obtaining an original user image and an original template image for the original user image; the original template image is from a passport photo template library; deforming the original template image according to the original user image and an image deformation algorithm to obtain a target template image for the original user image; inputting the original user image and the target template image into a trained image generation model, and fusing and adding noise to the face region and the non-face region of the original user image and the target template image respectively according to a dynamic weighting coefficient and a dynamic face mask to obtain a fused and noisy image; restoring the face region and the non-face region of the fused and noisy image respectively according to the dynamic weighting coefficient and the dynamic face mask to obtain a target passport photo; wherein the image deformation algorithm is obtained by constraint optimization on a thin plate spline (TPS) deformation algorithm, so that the target template image meets the requirements of face naturalness and passport photo specification at the same time, and the constraint includes one or more of face key point constraint, light and shadow consistency constraint and face structure constraint; the face key point constraint is used to indicate a dynamic weighting mechanism for the face key region and the non-face key region, so as to adopt different deformation weights to participate in image deformation processing; the image generation model is obtained by face feature reservation optimization training on a diffusion algorithm; the method for obtaining the dynamic weighting coefficient comprises: extracting features of the face region of the original user image to obtain a first feature vector, and extracting features of the target template image to obtain a second feature vector; calculating the similarity of the first feature vector and the second feature vector to obtain an image similarity; matching the positions of the face key points between the original user image and the target template image to obtain a face key point matching degree; weighting and summing the image similarity and the face key point matching degree, and performing normalization processing to obtain a dynamic weighting coefficient for the original user image and the target template image.

2. The method of claim 1, wherein, the deforming the original template image according to the original user image and an image deformation algorithm to obtain a target template image for the original user image comprises: detecting face key points of the original template image to obtain a plurality of original face key point positions, and detecting face key points of the original user image to obtain a plurality of user face key point positions; dynamically matching the user face key point positions to obtain a plurality of target face key point positions; nonlinearly deforming the face key region of the original template image and smoothly transitioning the non-face key region of the original template image according to the plurality of original face key point positions and the plurality of target face key point positions through the image deformation algorithm, and performing bilinear interpolation calculation to obtain a deformed template image that meets the face key point constraint; processing the deformed template image according to at least one of the light and shadow consistency constraint and the face structure constraint to obtain a target template image for the original user image.

3. The method of claim 2, wherein, The face key region of the original template image is nonlinearly deformed by an image deformation algorithm according to the plurality of original face key point positions and the plurality of target face key point positions, and the face non-key region of the original template image is smoothly transitioned, including: Obtaining the deformation weight corresponding to each of the different face key regions and face non-key regions; According to the deformation weight, the plurality of original face key point positions are mapped to the target face key point positions of the corresponding face key points to realize the nonlinear deformation of the face key region of the original template image and the smooth transition of the face non-key region, so as to minimize the distance between the original face key point positions and the target face key point positions corresponding to the same face key point; Wherein, the deformation weight corresponding to the face key point is greater than the deformation weight corresponding to the face non-key point; the deformation weight corresponding to the face key point with higher impact on the quality of the ID photo is greater.

4. The method of claim 2, wherein, According to the light and shadow consistency constraint, the deformed template image is processed, including: Obtaining the first light intensity of the original template image and the second light intensity of the deformed template image; According to the difference between the first light intensity and the second light intensity of the same pixel point, the light and shadow residual error between the original template image and the deformed template image is obtained; According to the light and shadow residual error and the light and shadow constraint weight, the deformed template image is smoothed to make the light and shadow effect of the processed template image consistent with the light and shadow effect of the original template image.

5. The method of claim 2, wherein, According to the face structure constraint, the deformed template image is processed, including: Determining the face symmetry line in the deformed template image; According to the face symmetry line and the symmetry constraint weight, the point position mirror transformation is performed on the deformed template image to make the face key points in the changed template image left-right symmetric about the face symmetry line.

6. The method according to any one of claims 1 to 5, characterized in that, The face key point positions between the original user image and the target template image are matched to obtain the face key point matching degree, including: Aligning the face center points of the original user image and the target template image; Calculating the relative average offset between the aligned original user image and the target template image to obtain the face key point matching degree between the original user image and the target template image.

7. The method according to any one of claims 1 to 5, characterized in that, According to the fusion weight and the dynamic face mask, the face region and the non-face region of the original user image and the target template image are fused and added with noise to obtain a fused and noisy image, and according to the generation weight and the dynamic face mask, the face region and the non-face region of the fused and noisy image are restored respectively to obtain a target ID photo, including: According to the dynamic weighting coefficient, the original user image and the target template image are fused to obtain a fused image; The face region of the fusion image is added with noise according to the dynamic face mask, and the non-face region of the target template image is gradually added with noise, the face region after being added with noise is fused with the non-face region after being added with noise, and a fusion noise-added image is obtained; The face region and the non-face region in the fusion noise-added image are gradually de-noised according to the first feature vector, the second feature vector and the dynamic weighting coefficient, and a target ID photo is obtained.

8. The method of claim 7, wherein, In the process of obtaining the image generation model by performing face feature reservation optimization training on the diffusion algorithm: According to the sample feature vectors of the sample fusion noise-added image, the sample user image and the sample template image, a feature consistency loss is obtained; According to the sample light and shadow intensity of the sample fusion noise-added image and the sample template image, a light and shadow consistency loss is obtained; According to the discriminator network, an adversarial loss between the predicted ID photo output by the initial image generation model and the standard ID photo is obtained; According to the predicted ID photo output by the initial image generation model and the standard ID photo, a reconstruction loss of the predicted ID photo is obtained; According to the feature consistency loss, the light and shadow consistency loss, the adversarial loss and the reconstruction loss, and the loss weights, a prediction total loss is obtained; The initial image generation model is trained by minimizing the prediction total loss, and an image generation model is obtained.

9. An electronic device, comprising: The electronic device comprises at least one communication element, at least one memory and at least one processor, wherein: The communication element is configured to receive an original user image and an original template image corresponding to the original user image, wherein the original template image is from an ID photo template library; The memory is configured to store a plurality of computer program instructions; The processor is configured to load and execute the computer program instructions to implement the following steps: According to the original user image and an image deformation algorithm, the original template image is deformed to obtain a target template image corresponding to the original user image; The original user image and the target template image are input into the trained image generation model, and the face region and the non-face region of the original user image and the target template image are fused and added with noise according to a dynamic weighting coefficient and a dynamic face mask to obtain a fusion noise-added image; According to the dynamic weighting coefficient and the dynamic face mask, the face region and the non-face region of the fusion noise-added image are restored to obtain a target ID photo. The image deformation algorithm is obtained by performing constraint optimization on a thin plate spline (TPS) deformation algorithm, so that the target template image meets the requirements of face naturalness and ID photo specification at the same time, and the constraint includes one or more of face key point constraint, light and shadow consistency constraint and face structure constraint. The image generation model is obtained by performing face feature reservation optimization training on a diffusion algorithm. The method for obtaining the dynamic weighting coefficient comprises: The face region of the original user image is extracted to obtain a first feature vector, and the target template image is extracted to obtain a second feature vector; performing similarity calculation on the first feature vector and the second feature vector to obtain an image similarity; performing matching on positions of facial key points between the original user image and the target template image to obtain a facial key point matching degree; performing weighted summation on the image similarity and the facial key point matching degree, and performing normalization processing to obtain a dynamic weighting coefficient for the original user image and the target template image.

Images