Identification photo generation method and electronic equipment

Abstract

The invention discloses an identification photo generation method and electronic equipment, and relates to the field of image processing, after an original user image and an original template image (identification photo template) aiming at the original user image are acquired, image deformation is carried out on the original template image according to the original user image and an image deformation algorithm, and the identification photo is generated. A target template image which aims at an original user image and conforms to one or more of facial key point constraints, light and shadow consistency constraints and facial structure constraints is obtained, and the original user image and the target template image are input into a trained image generation model (obtained by optimizing and training a diffusion algorithm); the corresponding areas are processed according to the different processing weights corresponding to the face area and the non-face area, it is ensured that the obtained target identification photo is more natural, real and reliable in vision, the specification requirement of an actual scene for the identification photo is met, the calculation time and cost are reduced, and the requirement for large-scale identification photo batch generation is met.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method and electronic device for generating ID photos. Background Art

[0002] As society's demand for digital and intelligent services continues to grow, people's needs for ID photos are becoming more diverse and personalized. However, traditional ID photo shoots typically require users to go to designated photo studios or related institutions and adhere to specific shooting requirements, such as background, clothing, and pose. This increases the complexity and cost of ID photo shoots, affecting user experience and convenience. Summary of the Invention

[0003] In view of the above problems, this application provides the following solutions: In a first aspect, the present application provides a method for generating an ID photo, the method comprising: Obtaining an original user image and an original template image corresponding to the original user image; the original template image is from a ID photo template library; Performing image deformation on the original template image according to the original user image and an image deformation algorithm to obtain a target template image for the original user image; Inputting the original user image and the target template image into a trained image generation model, processing the corresponding areas between the original user image and the target template image according to different processing weights corresponding to facial areas and non-facial areas, to obtain a target ID photo; The image deformation algorithm is obtained by constraining optimization of the thin plate spline TPS deformation algorithm, wherein the constraints include one or more of facial key point constraints, light and shadow consistency constraints, and facial structure constraints; The image generation model is obtained by optimizing and training the diffusion algorithm.

[0004] Optionally, performing image deformation on the original template image based on the original user image and an image deformation algorithm to obtain a target template image for the original user image includes: Performing facial key point detection on the original template image to obtain a plurality of original facial key point positions; and performing facial key point detection on the original user image to obtain a plurality of user facial key point positions; Dynamically matching the user's facial key point positions to obtain multiple target facial key point positions; According to the multiple original facial key point positions and the multiple target facial key point positions, nonlinearly deform the facial key area of ​​the original template image through an image deformation algorithm, and smoothly transition the facial non-key area of ​​the original template image, and obtain a deformed template image that satisfies the facial key point constraints through bilinear interpolation calculation; The deformed template image is processed according to at least one of the light and shadow consistency constraint and the facial structure constraint to obtain a target template image for the original user image.

[0005] Optionally, performing nonlinear deformation on the facial key area of ​​the original template image and performing smooth transition on the facial non-key area of ​​the original template image by an image deformation algorithm based on the multiple original facial key point positions and the multiple target facial key point positions, includes: Obtaining deformation weights corresponding to different facial key areas and non-key areas; According to the deformation weights, nonlinear deformation of the facial key area of ​​the original template image and smooth transition of the facial non-key area are achieved by mapping the multiple original facial key point positions to the target facial key point positions of the corresponding facial key points, so as to minimize the distance between the original facial key point positions and the target facial key point positions corresponding to the same facial key point; Among them, the deformation weight corresponding to the facial key point is greater than the deformation weight corresponding to the facial non-key point; the facial key point with a higher degree of influence on the quality of the ID photo has a greater deformation weight corresponding to it.

[0006] Optionally, processing the deformed template image according to the light and shadow consistency constraint includes: Acquire a first light and shadow intensity of the original template image and a second light and shadow intensity of the deformed template image; Obtaining a light and shadow residual between the original template image and the deformed template image according to a difference between the first light and shadow intensity and the second light and shadow intensity at the same pixel point; The deformed template image is smoothed according to the light and shadow residual and the light and shadow constraint weight, so that the light and shadow effect of the processed template image is consistent with the light and shadow effect of the original template image.

[0007] Optionally, processing the deformed template image according to the facial structure constraint includes: determining a facial symmetry line in the deformed template image; According to the facial symmetry line and the symmetry constraint weight, the deformed template image is subjected to point-wise mirror transformation so that the facial key points in the transformed template image are bilaterally symmetrical about the facial symmetry line.

[0008] Optionally, inputting the original user image and the target template image into a trained image generation model, processing corresponding areas between the original user image and the target template image based on different processing weights corresponding to facial areas and non-facial areas, to obtain a target ID photo, includes: Inputting the original user image and the target template image into a trained image fusion model, fusing and denoising the facial regions and non-facial regions of the original user image and the target template image respectively according to the fusion weight and the dynamic facial mask, to obtain a fused noisy image; The facial region and the non-facial region of the fused and noisy image are restored respectively according to the generated weights and the dynamic facial mask to obtain a target ID photo.

[0009] Optionally, the fusion weight and the generated weight are the same dynamic weighting coefficient, and the method for obtaining the dynamic weighting coefficient includes: Performing feature extraction on the facial region of the original user image to obtain a first feature vector, and performing feature extraction on the target template image to obtain a second feature vector; performing similarity calculation on the first eigenvector and the second eigenvector to obtain image similarity; Matching the facial key point positions between the original user image and the target template image to obtain a facial key point matching degree; The image similarity and the facial key point matching degree are weighted and summed, and normalized to obtain dynamic weighting coefficients for the original user image and the target template image.

[0010] Optionally, matching the facial key point positions between the original user image and the target template image to obtain a facial key point matching degree includes: Aligning the facial center points of the original user image and the target template image; The relative average offset between the aligned original user image and the target template image is calculated to obtain the facial key point matching degree between the original user image and the target template image.

[0011] Optionally, the fusing and denoising the facial region and non-facial region of the original user image and the target template image based on the fusion weights and the dynamic facial mask to obtain a fused noisy image, and restoring the facial region and non-facial region of the fused noisy image based on the generated weights and the dynamic facial mask to obtain a target ID photo, including: fusing the original user image and the target template image according to the dynamic weighting coefficient to obtain a fused image; Noising the facial region of the fused image and gradually noising the non-facial region of the target template image based on the dynamic facial mask, fusing the noisy facial region with the noisy non-facial region to obtain a fused noisy image; Based on the first eigenvector, the second eigenvector and the dynamic weighting coefficient, the facial area and the non-facial area in the fused noisy image are gradually denoised to obtain a target ID photo.

[0012] Optionally, in the process of obtaining the image generation model by optimizing and training the diffusion algorithm: Obtain feature consistency loss based on the sample feature vectors of the sample fusion noisy image, the sample user image, and the sample template image; Obtain the light and shadow consistency loss based on the sample light and shadow intensities of the sample fusion noisy image and the sample template image; Based on the discriminator network, obtain the adversarial loss between the predicted ID photo output by the initial image generation model and the standard ID photo; Obtaining a reconstruction loss of the predicted ID photo based on the predicted ID photo and the standard ID photo output by the initial image generation model; Obtaining a total prediction loss according to the feature consistency loss, the light and shadow consistency loss, the adversarial loss, the reconstruction loss, and the weights of each loss; The initial image generation model is trained by minimizing the total prediction loss to obtain an image generation model.

[0013] A second aspect of the present application provides an electronic device, comprising: at least one communication element, at least one memory, and at least one processor, wherein: The communication element is configured to receive an original user image and an original template image corresponding to the original user image; the original template image is from a ID photo template library; The memory is used to store a plurality of computer program instructions; The processor is configured to load and execute the computer program instructions to implement the following steps: Performing image deformation on the original template image according to the original user image and an image deformation algorithm to obtain a target template image for the original user image; Inputting the original user image and the target template image into a trained image generation model, processing the corresponding areas between the original user image and the target template image according to different processing weights corresponding to facial areas and non-facial areas, to obtain a target ID photo; Among them, the image deformation algorithm is obtained by constrained optimization of the thin plate spline TPS deformation algorithm, and the constraints include one or more of facial key point constraints, light and shadow consistency constraints, and facial structure constraints; the image generation model is obtained by optimizing and training the diffusion algorithm.

[0014] It can be seen that in the ID photo generation method provided by the present application, after obtaining the original user image and the original template image (ID photo template) for the original user image, the original template image is deformed according to the original user image and the image deformation algorithm to obtain a target template image for the original user image that meets one or more of the facial key point constraints, light and shadow consistency constraints, and facial structure constraints. Only then will the original user image and the target template image be input into the trained image generation model (obtained by optimizing the diffusion algorithm for training), and the corresponding areas are processed according to the different processing weights corresponding to the facial area and the non-facial area, to ensure that the obtained target ID photo is more natural and realistic visually, and reliably meets the standard requirements for ID photos in actual scenarios, and reduces the computing time and cost to meet the large-scale batch generation of ID photos. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0016] Figure 1 This is a flow chart of the ID photo generation method proposed in Example 1 of this application; Figure 2 This is a flowchart of the ID photo generation method proposed in Example 2 of this application; Figure 3 This is a flowchart of the ID photo generation method proposed in Example 4 of this application; Figure 4 This is a schematic diagram of the deformation weights of different facial regions in the template image in the ID photo generation method proposed in this application; Figure 5 This is a flowchart of the ID photo generation method proposed in Example 4 of this application; Figure 6 This is a schematic diagram of the structure of a device for generating ID photos proposed in an embodiment of the present application; Figure 7 A schematic diagram of the hardware structure of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation section of this application are only used to explain the specific embodiments of the present application and are not intended to limit this application. The embodiments of the present application will be described below in conjunction with the drawings. It will be known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0018] Moreover, the terms "first", "second" etc. in the full description of the application and the accompanying drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0019] With the significant progress of artificial intelligence (AI) technology in facial recognition and image generation in recent years, compared with using traditional image processing software such as Adobe Photoshop (PS) to manually edit and modify the captured user image to obtain the required ID photo, it is proposed to use various online ID photo generation tools such as Nice ID Assistant, Simple AI and Picsman (online ID photo production tool), or open source ID photo generation tools such as IDIFY (HivisionIDPhotos) or AI-IDPhoto. After uploading the user image and selecting the ID photo template, the user ID photo matching the ID photo template will be automatically generated. No manual operation is required by the user, which is simpler and more convenient, saves costs, and improves the efficiency of ID photo generation.

[0020] However, when using the aforementioned ID photo generation tools to automatically generate user ID photos, deep learning-based image segmentation and key point detection techniques are typically used to identify the facial area of ​​the user image, extract key point features from the facial area, and fuse them with the ID photo template to produce an ID photo that meets the template's clothing, hairstyle, and background requirements. However, this approach can easily lead to unnatural fusion between the user's facial contours and the ID photo template, resulting in unnatural details in the generated ID photo, which can easily create an unrealistic visual experience. It is also possible that the ID photo template may not meet the standard requirements for standard ID photos, due to factors such as uneven lighting and background interference, resulting in the generated ID photo failing to meet the strict requirements for standardized ID photos in various scenarios.

[0021] In order to further improve the above-mentioned problems, the present application proposes a new method for generating ID photos. The ID photo generation method proposed in the embodiment of the present application will be introduced in detail below with reference to the accompanying drawings.

[0022] Reference Figure 1 , is a flow chart of a method for generating a certificate photo according to the first embodiment of the present application. The method can be applied to an electronic device, which can be a server or a terminal device with data processing capabilities, such as Figure 1 As shown, the ID photo generation method proposed in this embodiment includes but is not limited to the following steps: Step S11, obtaining an original user image and an original template image corresponding to the original user image; the original template image comes from a ID photo template library; In an embodiment of the present application, the original user image can be a frame of image containing at least the user's head, which is collected by an image collector integrated into or external to the electronic device. In scenarios where ID photos are generated, such as online identity authentication, remote office, online education, corporate onboarding processes, or online ID processing, the image collector can be started to shoot the user's head to obtain one or more frames of clear original user images. During the shooting process, corresponding shooting prompts can also be output according to the ID photo specifications in the corresponding scenario to prompt the user to adjust the shooting angle, environment, or action / posture, etc., to obtain a high-quality original user image. This application does not limit the implementation method of online collection of original user images.

[0023] Of course, this application can also select one or more original user images that can be used to generate ID photos from the local storage space of the electronic device or other storage devices. This application does not limit the source of the original user images. In the case where this application obtains multiple original user images, before executing subsequent steps, one original user image can be selected through comparison and analysis to participate in the subsequent ID photo generation step, and the other original user images are not processed. Alternatively, each original user image obtained can participate in the subsequent ID photo generation step, and then the most suitable target ID photo can be selected from the corresponding ID photos generated. The ID photo generation step process for each original user image is the same, and this application will not describe them in detail one by one.

[0024] In the different ID photo generation scenarios listed above, the standard requirements for standardized ID photos often vary, such as background color, user clothing or hairstyle, etc. Therefore, when determining the target type of ID photo to be generated for the original user image, such as an ID card, student ID, visa application, or work ID, the user can manually select a matching ID photo template from the standardized ID photo template library in accordance with the corresponding standard requirements to determine it as the original template image for the original user image. That is, in response to the selection operation of multiple ID photo templates of the target type output, the original template image for the original user image is determined. The implementation process is not described in detail in this application.

[0025] In one possible implementation, the electronic device may also use an image search engine to search for an ID photo template from an ID photo template library that matches the scene where the original user image is located or the ID photo conditions input, and determine it as the original template image. For example, scene information recognition is performed on the scene where the original user image collected online is located, and based on the recognized scene information, an original template image that meets the ID photo specification requirements under the scene information is selected from the ID photo template set belonging to the matching target type. Alternatively, the ID photo conditions input by the user (such as mouse keys, joysticks, or voice input methods) are obtained, and the ID photo template library is searched for the ID photo template with the highest degree of matching with the ID photo conditions as the original template image, etc. This application does not limit the implementation method of the original template image, and it can be determined according to the circumstances.

[0026] Step S12: Based on the original user image and an image deformation algorithm, the original template image is deformed to obtain a target template image for the original user image; the image deformation algorithm is obtained by constraining optimization of a thin plate spline (TPS) deformation algorithm, wherein the constraints include one or more of facial key point constraints, light and shadow consistency constraints, and facial structure constraints; As described above, the original template image is directly selected from the ID photo template library. The ID photo features presented are fixed and single, and are likely to have a low degree of match with the standard requirements of standard ID photos in actual scenarios. As a result, the ID photos directly generated based on this cannot meet the strict requirements of the actual scenario for standardized ID photos, and a lot of time and effort are required to edit and modify the generated ID photos.

[0027] To address this issue, this application proposes to dynamically adjust the original template image. While preserving lighting and shadow effects, this approach achieves high-precision deformation of key facial regions (such as the eyes, nose tip, mouth corners, and chin). This ensures that these deformed key facial regions conform to the standards for standard ID photos in real-world scenarios, while allowing for a flexible and smooth transition to non-critical facial regions to preserve their characteristics. The facial region structure can also be adjusted as needed to achieve bilateral symmetry, ensuring a natural-looking face in the resulting target ID photo.

[0028] In an embodiment of the present application, an improved TPS (Thin Plate Spline) deformation algorithm can be used to perform an affine transformation on the original template image based on the facial key point features in the original user image, such as implementing one or more transformation operations such as overall scaling, translation, and rotation of the facial area in the original template image, and quickly matching the approximate shape and position of the face in the original user image. In this way, personalized matching of different facial features (such as round face, square face, or long face, etc.) can be achieved. In combination with other constraints, the local part of the original template image (i.e., the facial key area) can also be finely deformed to solve its detail matching and contour distortion problems, so that the deformed template image perfectly fits the face shape in the original user image, while retaining the skin texture and light and shadow effects in the original template image. It can also constrain the left-right symmetry of the face in the deformed template image to make its visual effect coordinated and consistent.

[0029] It can be seen that for different original user images, even if the same original template image is selected, the target template image obtained according to the image deformation method described above may also be different, so that it not only meets the standard requirements of standardized ID photos, but also achieves high-quality, natural fit and personalized visual presentation, which is then used to implement subsequent ID photo generation steps. Compared with directly using a fixed original template image, this application improves the diversity of ID photo templates through this dynamic adjustment of template image processing method, meets personalized needs, and can adapt to a variety of lighting, background and resolution conditions, ensuring that the generated ID photos meet the application standards of multiple scenarios, greatly improving the user experience and satisfaction.

[0030] It should be noted that the constraints of the image deformation algorithm used in this application can be flexibly adjusted or modified according to the actual scene requirements and are not limited to the constraints listed above. Combined with the above analysis, the facial key point constraint is used to indicate the dynamic weighting mechanism for facial key areas and non-facial key areas, so as to use different deformation weights to participate in the image deformation process. The light and shadow consistency constraint is used to ensure that the light and shadow effects of the template image before and after deformation are consistent. The facial structure constraint is used to indicate the symmetry of the left and right facial structures in the deformed template image. As for how to implement these constraints, this application does not limit it.

[0031] In step S13, the original user image and the target template image are input into a trained image generation model, and the corresponding areas between the original user image and the target template image are processed according to the different processing weights corresponding to the facial area and the non-facial area to obtain the target ID photo; the image generation model is obtained by optimizing and training the diffusion algorithm.

[0032] Following the above analysis, the target template image fully meets the standard requirements of standardized ID photos in actual scenarios, and solves the problems of distortion of key facial areas, light and shadow disharmony, and facial asymmetry in the original template image. The target template image and the original user image are input into the image generation model for processing to ensure that the output target ID photo meets the standard requirements of the standardized ID photo.

[0033] Because the image generation model of this application is optimized and trained using a diffusion algorithm, it can be an optimized diffusion model. Whether in the forward denoising process or the reverse denoising process, different processing weights are assigned to the facial and non-facial regions of the image. This allows the image generation model to process these two regions to different degrees, preserving the user's facial features and the template's light and shadow texture in the facial region, ensuring the generation of a natural, smooth, and high-quality professional-grade target ID photo. This application does not describe the forward denoising and reverse denoising implementation processes in detail.

[0034] In summary, in the scenario of automatically generating a user's ID photo, the present application selects an ID photo template, i.e., an original template image, that meets the requirements of the ID photo scenario. Then, the image deformation algorithm is used to deform the image to obtain a target template image that meets the requirements of the standard ID photo in the current ID photo scenario, i.e., dynamically adjust the ID photo template image. Since the image deformation algorithm is obtained by optimizing the TPS deformation algorithm by one or more of the following constraints, such as facial key point constraints, light and shadow consistency constraints, and facial structure constraints, the adjusted target template image and the original user image are input into the image generation model, i.e., the optimized and trained diffusion model, to reliably and quickly generate a target ID photo that meets the requirements of the standard ID photo, so that the target ID photo can accurately retain the skin texture, light and shadow effects, and background details of the target template image, while maintaining the similarity of the user's facial features, making the target ID photo more natural and realistic visually, and reliably meeting the standard requirements of the ID photo in actual scenarios. In addition, the present application effectively shortens the calculation time and reduces the consumption of computing resources through the denoising and restoration process of the diffusion model, so that the present application can be applied to ordinary equipment with limited resources and meet the needs of large-scale batch generation of ID photos.

[0035] Reference Figure 2 , is a flow chart of the ID photo generation method proposed in the second embodiment of the present application. This embodiment can describe an optional implementation process of how to deform the original template image to obtain the target template image in the above ID photo generation method, such as Figure 2 As shown, the implementation process may include but is not limited to the following steps: Step S21, performing facial key point detection on the original template image to obtain multiple original facial key point positions, and performing facial key point detection on the original user image to obtain multiple user facial key point positions; Step S22, dynamically matching the user's facial key point positions to obtain multiple target facial key point positions; Combined with the above description of the technical solution of this application, in order to reduce facial matching errors, this application will use high-precision facial feature point positioning and geometric transformation to determine the precise matching of the user's facial features in the original user image with the facial key points of the ID photo template image, avoiding facial contour deviation and unnatural phenomena, so as to improve the accuracy and authenticity of the subsequent ID photo generation. To this end, this application can perform facial key point detection on the original template image and the original user image respectively to determine the positions of their respective facial key points, and generate the target facial key point positions by dynamically matching the detected user facial key points to achieve subsequent matching deformation processing.

[0036] In some embodiments, the present application may use constrained local models (CLM) or active shape models (ASM) or active appearance models (AAM) to locate facial key points. Other key point detection models trained by deep learning methods may also be used to identify facial key points in an image. The present application does not limit the facial key point detection method. For the convenience of description, the original template image may be denoted as I m , the coordinates of the positions of the original facial key points detected can be expressed as (x i ,y i ), therefore, the original facial key point set P composed of all original facial key point positions m It can be expressed as: m ={(x i ,y i )}.

[0037] According to the facial key point detection method described above, based on the determination of the user's facial key points in the original user image, the user's facial key points can be adjusted to the key points of the target posture or shape / expression, i.e., the target facial key points, through dynamic matching or optimization algorithms. The target posture or shape / expression can be determined based on the actual scene. After that, the position of the user's facial key points can be adjusted through a dynamic matching method based on interpolation, optimization or deep learning to obtain the corresponding multiple target facial key point positions, which constitute a set, the target facial key point set. It can be expressed as: . Preferably, the present application can maintain the high-resolution feature map of the input image (original user image) through a high-resolution network (HRNet), improve the key point positioning accuracy, support multi-face detection, be robust to occlusion and extreme postures, and reduce the requirements for shooting the original user image.

[0038] Among them, interpolation-based dynamic matching is to gradually adjust the position of the user's facial key points to the target facial key point position through interpolation methods (such as linear interpolation or spline interpolation); optimization-based dynamic matching is to minimize the position difference between the user's facial key points and the target facial key points through optimization algorithms (such as gradient descent); deep learning-based dynamic matching is to use a deep learning model (such as a generative adversarial network or a transformer) to learn the mapping relationship between the user's facial key points and the target facial key points, so as to use the trained model to map the actually detected user's facial key point positions to the target facial key point positions. The implementation process is not described in detail in this application.

[0039] In addition, in practical applications, the number of facial key points detected can be 98 key points, 68 key points, 29 key points, or 21 key points, etc., which can be determined according to actual needs. In this application, the number of original facial key point positions and the number of target facial key point positions can be the same, that is, P m and Both collections have the same number of elements.

[0040] Step S23, based on the multiple original facial key point positions and the multiple target facial key point positions, nonlinearly deforming the facial key area of ​​the original template image using an image deformation algorithm, and smoothly transitioning the facial non-key area of ​​the original template image, and performing bilinear interpolation calculation to obtain a deformed template image that satisfies the facial key point constraints; In the embodiment of the present application, the target deformation function for the original template image can be obtained by minimizing the image deformation algorithm based on the positions of multiple original facial key points and multiple target facial key points, and then the original template image I can be transformed into the target template image according to the target deformation function. m Each pixel point (x, y) in is mapped to the corresponding target position , after bilinear interpolation calculation, the deformed template image is obtained In the solution process using the image deformation algorithm, the distance between the original facial key point position and the target facial key point position corresponding to the same facial key point can be minimized, and combined with the following constraints, the deformed template image is optimized to obtain a target template image that better meets the standard requirements of standardized ID photos in actual scenarios. This application does not elaborate on the solution process of the improved TPS deformation algorithm.

[0041] In one possible implementation, when determining P m and Afterwards, the TPS deformation algorithm can be applied to transform P m The original facial key point positions in are mapped to The target facial key point positions of the corresponding facial key points in the original template image are used to achieve nonlinear flexible deformation of the facial key areas in the original template image (i.e., areas where pre-defined facial key points are located, such as eyes, nose tip, corners of mouth, cheeks, forehead, mandible and chin, etc.). Combined with this mapping relationship, a smooth transition is performed on the non-critical facial areas in the original template image (such as the outer edge of the face and the background transition area, etc.), thereby improving the facial shape fit. As analyzed above, this application can use the deformation function in the image deformation algorithm to map each pixel point (x, y) in the original template image to a new coordinate, i.e., the target position. , but there are non-integer coordinate points in the target position. By using the integer coordinate points of the pixels around the target position, the pixel values ​​corresponding to the template image after the above mapping deformation processing can be calculated by bilinear interpolation, which is recorded as The calculation process is not described in detail in this application.

[0042] Step S24 : Processing the deformed template image according to at least one of the light and shadow consistency constraint and the facial structure constraint to obtain a target template image for the original user image.

[0043] When deforming the original template image using an image deformation algorithm, to ensure a natural transition between light and shadow effects and skin texture, the difference in light and shadow effects before and after deformation can be added to the deformed template image based on the light and shadow consistency constraints in the image deformation algorithm. This ensures that the deformed template image retains the light and shadow effects of the original template image, thereby ensuring the harmony and natural transition of light and shadow in the deformed template image. Optionally, this light and shadow effect difference can be calculated based on parameters in the template image that affect the light and shadow effects, such as the difference in light and shadow intensity before and after deformation. This application does not describe this calculation process in detail.

[0044] Preferably, in order to further improve the overall visual effect of the deformed template image and to improve the visual effect of the generated ID photo, in the process of deforming the original template image using the image deformation algorithm, the mapping of the left and right facial key point positions in the template image can also be smoothly constrained based on the facial structure constraints in the image deformation algorithm, such as facial symmetry constraints, to ensure that the left and right structures of the facial area in the generated image are symmetrical.

[0045] Based on this, in some embodiments, a facial alignment method based on affine transformation of facial key points can be adopted, and symmetrical key points can be generated based on a deep learning model. It can also be implemented on the basis of facial alignment through mirror symmetry adjustment or symmetry detection and correction. It can also be implemented based on predefined symmetry templates or dynamic symmetry adjustment. This application does not limit the implementation method of how to achieve symmetrical constrained deformation of the facial area during the deformation process of the original template image.

[0046] Reference Figure 3 , is a flow chart of the ID photo generation method proposed in Example 3 of this application. This embodiment can describe another optional refinement method for how to deform the original template image to obtain the target template image in the above ID photo generation method, such as Figure 3 As shown, the implementation process may include but is not limited to the following steps: Step S31, obtaining a plurality of original facial key point positions of the original template image and a plurality of corresponding target facial key point positions; Regarding the implementation method of step S31, reference may be made to the description of the corresponding part of the above embodiment, which will not be elaborated in this embodiment.

[0047] Step S32: Obtain deformation weights corresponding to different facial key points and non-key points. The deformation weights corresponding to facial key points are greater than the deformation weights corresponding to non-key points, and the deformation weights corresponding to facial key points that have a greater impact on the quality of the ID photo are greater. In the embodiment of the present application, the facial area is divided into key facial areas and non-key facial areas based on the different importance (i.e., influence) of different points in the facial area for generating high-quality ID photos. The key facial areas can be further divided into fine-grained areas, such as the eye area, nose tip area, mouth corner area, cheek area, forehead area, and mandible and chin area. At the same time, different deformation weights can be configured for each facial area based on the different importance levels to indicate the degree of deformation of the corresponding facial area during the deformation of the original template image, thereby avoiding deformation distortion. The correspondence between different importance levels and deformation weights of different facial areas can also be determined based on experience or historical data, or can be implemented using other weight dynamic allocation methods, and this application does not impose any restrictions on this.

[0048] In a possible implementation, each facial region divided according to the above example is subjected to a pre-configured deformation weight w based on a dynamic weighting mechanism. i You can Figure 4 The contents of the table shown need to be explained. The values ​​of the deformation weights corresponding to each facial area include but are not limited to Figure 4 The sizes shown and the types of facial area divisions include but are not limited to Figure 4 The content shown can be adaptively adjusted according to the actual scene requirements, and this application will not be detailed here. Based on this, in the actual application of this embodiment, you can use Figure 4 The data can be stored in a table format as shown, or after determining the position range of each facial region, a piecewise function is generated based on the position range and the deformation weight, so that it can be directly read. Of course, this application can also configure the corresponding deformation weight for each facial region online according to the pre-learned dynamic weight allocation rule, but this application does not describe the implementation process in detail.

[0049] Step S33, mapping the multiple original facial key point positions to the target facial key point positions of the corresponding facial key points according to different deformation weights, thereby achieving nonlinear deformation of the facial key area of ​​the original template image and smooth transition of the facial non-key area, so as to minimize the distance between the original facial key point position and the target facial key point position corresponding to the same facial key point; In the image deformation algorithm proposed in this application, namely the improved TPS deformation algorithm, different deformation weights can be applied to key facial areas and non-key facial areas through a dynamic area weighting mechanism to ensure high-precision matching of key facial areas during template image deformation, while making smooth transitions to non-key facial areas to avoid deformation distortion. Among them, since different key facial areas have different effects on the quality of ID photos, this application also configures different deformation weights, such as Figure 4 As shown, the high-precision deformation of different key facial areas is further improved to ensure natural facial fit.

[0050] Based on the above analysis, the deformation function (i.e., interpolation function) obtained by solving the image deformation algorithm is used to perform an affine transformation on the original template image. In this implementation process, in the facial key point constraint term of its bending energy function, the application adopts a facial dynamic weighting mechanism to assign different deformation weights w to different facial regions in order to ensure that the deformed facial key points are as close as possible to the corresponding target key points. i , this part can be expressed as: ; (1) in, represents the target position (new coordinates) of the i-th target facial key point, Indicates the position of the i-th original facial key point after deformation. i=1, 2, ..., n, where n represents the number of facial key points, such as 98, etc. This application does not limit its value and it can be determined according to the situation. It can be seen that in the process of mapping each pixel point in the original template image to the new coordinate, i.e., the target position, the deformation weight w of each pixel point position belonging to different facial key areas and non-key areas of the face is i Different, so that the deformation degree of the corresponding position is different, so as to better meet the pixel position deformation requirements of facial areas at different positions, to minimize the distance between the original facial key point position and the target facial key point position corresponding to the same facial key point, to ensure high-precision matching of facial key areas, and smooth transition of non-critical facial areas to avoid deformation and distortion.

[0051] It should be understood that, in conjunction with the operational principles of the TPS algorithm, when deforming the original template image, the curvature of the deformation function can be constrained by the second-order partial derivative, making the deformation smooth and natural. Therefore, the curvature energy function of the image deformation algorithm also includes the following smoothing regularization term: ; (2) Among them, λ can represent a smoothing regularization parameter used to adjust the constraint strength. Its value can be determined according to actual needs. This application does not limit the method for determining the value of λ and its size.

[0052] Combined with the above analysis, in the process of deforming the original template image, after the facial key point constraint and smoothing regularization processing, the pixel value of the pixel point in the deformed template image can be expressed as: , as analyzed above, the pixel value can be calculated by bilinear interpolation method: ; (3) In formula (3), i = 0, 1, ..., m, where m represents the width pixel value of the original template image; j = 0, 1, ..., n, where n represents the height pixel value of the image. For example, if the size of the original template image is 640 × 720, m = 640 and n = 720. represents the original pixel value of the pixel in the i-th row and j-th column of the original template image, w ij It can represent the interpolation weight of the pixel in the i-th row and j-th column, which can be determined by the distance between the original coordinates and the target coordinates of the pixel. Optionally, in bilinear interpolation, the interpolation weight varies inversely with the distance. The closer the original coordinates of the pixel in the original template image are to the target coordinates (i.e., the smaller the distance), the greater the interpolation weight. The interpolation weight can be calculated using, but is not limited to, the following methods: ; (4) Among them, (x i ,y i ) represents mapping the original coordinates (x, y) of the pixel point to the target position Finally, adjust the non-integer coordinates to integer coordinates.

[0053] Step S34, obtaining a first light and shadow intensity of the original template image and a second light and shadow intensity of the deformed template image; Step S35, obtaining a light and shadow residual between the original template image and the deformed template image based on the difference between the first light and shadow intensity and the second light and shadow intensity at the same pixel point; Step S36, smoothing the deformed template image according to the light and shadow residual and the light and shadow constraint weight, so that the light and shadow effect of the processed template image is consistent with that of the original template image; As analyzed above, in addition to considering facial keypoint constraints, this application also considers light and shadow consistency during the deformation of the original template image. This ensures that the light and shadow effects of the template image before and after deformation are consistent, ensuring that the light and shadow in the deformed template image are consistent with the skin texture. In one possible implementation, this can be achieved by calculating the light and shadow residual to maintain the light and shadow effects of the template image.

[0054] Based on this, we can refer to the light and shadow consistency constraint R expressed by the following formula light , get the light and shadow residual: ; (5) In formula (5), P i Represents the position (coordinate value) of the i-th original facial key point in the original template image. It can represent the light and shadow intensity at this position, recorded as the first light and shadow intensity, It can represent the target position of the i-th target facial key point in the deformed template image. The light and shadow intensity of the target position can be expressed as the second light and shadow intensity. The difference between the first and second light and shadow intensities corresponding to the same facial key point is calculated to obtain the difference between the original template light and shadow of the corresponding facial key point and the target template light and shadow, that is, the light and shadow intensity difference. The square sum of the light and shadow intensity differences of each pixel is then used as the light and shadow consistency constraint term R of the bending energy function in the image deformation algorithm. light , used to realize the calculation of deformation function.

[0055] In addition, for R light A corresponding control weight, namely the light and shadow constraint weight, can also be configured to control the importance of the light and shadow consistency constraint. It can be denoted as γ, and its value range can be 0.01~0.1, etc. It can be flexibly configured or adjusted according to the importance of the light and shadow consistency constraint in the actual scene. Based on this, the present application can add the light and shadow residuals of each pixel point mentioned above to the deformed template image, such as adding the light and shadow residuals of each pixel point to the pixel value of the corresponding pixel point, so that the light and shadow effects of the template image before and after deformation are consistent.

[0056] Step S37, determining the facial symmetry line in the deformed template image; Step S38: performing a point-wise mirror transformation on the deformed template image according to the facial symmetry line and the symmetry constraint weight, so that the facial key points in the transformed template image are bilaterally symmetrical about the facial symmetry line.

[0057] During the deformation process of the original template image, the facial region of the original template image can also be symmetric optimized according to the facial structure constraint. Combined with the above description of the facial structure constraint, this application requires that the key points of the facial region are symmetrical to ensure the left-right symmetry of the facial structure. This embodiment can use the following formula to implement the facial structure constraint R symmetry : ; (6) In formula (6), Sym() can represent a symmetric transformation operation such as point mirror transformation to ensure that the deformed facial key points are symmetrical about the facial symmetry line (such as the facial center axis). left Represents the key point set of the left side of the face (left face), P rightrepresents the set of facial key points on the right side of the face (right face). It should be understood that the expression of facial structure constraint P in formula (6) left and P right The position of the pixels / key points in this application can be replaced. The pixel points / key points in this application can be represented by their coordinate values ​​(positions) in the image. Therefore, the key point set of this application refers to the key point position set, which will not be explained one by one in the following text.

[0058] In the actual application of this embodiment, the center points of the key facial areas can be used to construct the facial symmetry line, that is, the facial center axis. For example, the center point of the left eye, the center point of the right eye, the center point of the nose tip, the center point of the forehead, the center point of the lower lip, the center point of the chin, and other key points can be selected. Then, the horizontal center axis can be calculated separately, such as calculating the average value C of the x-coordinates of the selected facial key points. x , determine the position of the vertical axle, and then determine the vertical center line equation to represent the vertical center axis, that is, x=C x , that is, a straight line perpendicular to the x-axis. Based on this, for any facial key point P(x, y) in the original template image, its distance from the central axis (x=C x ) Symmetrical point ,in, , =2 C x -x.

[0059] In this way, this embodiment can realize the point mirror transformation of the facial area in the deformed template image in this way and determine the pixel value of the corresponding pixel point after the transformation. As analyzed above, after the mapping of each pixel point in the original template image is realized based on the facial key point constraint, the pixel value obtained by bilinear interpolation calculation is recorded as , after the above method, the light and shadow residual smoothing is introduced into the deformed template image for compensation optimization, and the pixel value of the corresponding pixel point can be recorded as After optimizing the facial symmetry of the deformed template image, the following formula can be used to express the facial symmetry: To update: ; (7) In formula (7), the content on the left side of the equal sign represents the pixel value of the pixel point in the final generated target template image, and the numerator on the right side of the equal sign represents the pixel value after light and shadow consistency optimization and facial structure symmetry optimization, and the average value is taken as the final pixel value. This approach ensures that the target template image is visually smooth and natural, with natural transitions in lighting and texture, and that the facial area is symmetrical. This allows for high-precision facial fitting and preserves the light and shadow effects and skin texture of the template image for ID photos.

[0060] The aforementioned point-to-point mirroring process can be combined with a symmetry constraint weight δ for the facial region. This δ can typically be between 0.05 and 0.2, with no specific value restrictions. It should be noted that methods for achieving bilateral facial symmetry include, but are not limited to, the calculation methods listed above.

[0061] Based on the above analysis, the bending energy function of the image deformation algorithm proposed in this application, that is, the bending energy function of the improved TPS deformation algorithm can be expressed as: ; (8) ; (9) From formula (9), we can see that compared with the bending energy function of the traditional TPS deformation algorithm, this application optimizes the facial key point constraints and introduces deformation weights w for different facial regions. i , to ensure high-precision deformation of key facial areas and flexible and smooth transition of non-key facial areas. Compared with the treatment of each facial area with the same degree of deformation, the dynamic area weighting mechanism of this application can effectively solve the problems of detail matching and contour distortion. And as shown in formula (8), in order to ensure that the deformed template image perfectly fits the original user image face shape while retaining the matching texture and light and shadow effects of the template image, this application also introduces the light and shadow consistency constraint R of the template image light and facial structure constraint R symmetry .

[0062] In practical applications, the bending energy function E final (f) is solved to obtain the deformation function (i.e., interpolation function) f, which can be expressed as f(x, y). Since the facial key point constraints, smoothing regularization, light and shadow consistency constraints, and facial structure constraints are taken into account in the solution process, the deformation function f(x, y) is used to map each pixel point in the original template image to the target position. After obtaining the deformed template image, the deformed pixel points are interpolated according to the mapping relationship obtained thereby to ensure that the facial area of ​​the generated target template image fits the user's face shape, solve the problems of detail matching and contour distortion, and achieve a natural transition between light and shadow and skin texture, and a left-right symmetry of the facial area, thereby improving the overall visual effect.

[0063] Reference Figure 5 , is a flow chart of the ID photo generation method proposed in the fourth embodiment of the present application. This embodiment can provide a detailed description of an optional implementation process of how to obtain the target ID photo by inputting the original user image and the target template image into the trained image generation model in the above ID photo generation method, such as Figure 5 As shown, the refined implementation method includes: Step S51: Input the original user image and the target template image into the trained image fusion model, perform feature extraction on the facial area of ​​the original user image to obtain a first feature vector, and perform feature extraction on the target template image to obtain a second feature vector; The original user image I p and the target template image After inputting the trained image fusion model, in order to control the fusion ratio of the original user image and the target template image, the generated ID photo can be personalized and standardized dynamically. To this end, the application can combine the eigenvectors of the input images to calculate dynamic weighting coefficients, which serve as the fusion weights of the two input images in the forward denoising process and the restoration weights of the two images in the reverse denoising process.

[0064] In order to obtain a suitable dynamic weighting coefficient, the input image can be extracted through a pre-trained feature extraction network (such as the feature extraction model based on InsightFace) to obtain the corresponding feature vector. Among them, the feature extraction of the original user image is to extract the features of its facial area, that is, to extract the key facial area features used to represent the user in the original user image, that is, the facial features, such as the position and shape of the key facial areas such as eyes, nose and mouth, etc. The feature vector thus extracted is recorded as the first feature vector F user For the target template image, feature extraction can be performed directly to obtain the second feature vector F template The implementation process will not be described in detail in this application.

[0065] Step S52, performing similarity calculation on the first eigenvector and the second eigenvector to obtain image similarity; Step S53, matching the facial key point positions between the original user image and the target template image to obtain the facial key point matching degree; In the embodiment of the present application, a similarity calculation method such as the cosine similarity algorithm or the Euclidean distance can be used to calculate the similarity between the first eigenvector and the second eigenvector to obtain the image similarity, which can be recorded as Sim(F user , F template ). This similarity calculation method or other key point matching algorithms can also be used to implement step S53. Taking the Match() key point matching algorithm as an example, the original user image and the target template image can be aligned with the facial center point, and then the relative average offset between the aligned original user image and the target template image is calculated to obtain the facial key point matching degree between the original user image and the target template image, which can be expressed as Match(P user , P template ).

[0066] Step S54: performing a weighted summation of the image similarity and the facial key point matching degree, and performing normalization processing to obtain dynamic weighting coefficients for the original user image and the target template image; Based on the above analysis, the dynamic weighting coefficient λ can be calculated using the following formula: ; (10) In this embodiment, the Sigmoid function can be used to implement normalization processing. For the image similarity and facial key point matching obtained above, as shown in formula (10), corresponding weighting coefficients, α and β, can be configured to control the importance of image similarity and facial key point matching. This application does not limit the values ​​of these two weighting coefficients, and they can be configured or adjusted based on experience or historical data.

[0067] Step S55, fusing and denoising the facial region and non-facial region of the original user image and the target template image respectively according to the dynamic weighting coefficient and the dynamic facial mask, to obtain a fused noisy image; In the embodiment of the present application, a dynamic facial mask M can be constructed. face , used to distinguish the facial area and non-facial area in the image, such as M face The area with the element value of 1 in represents the face area, and the area with the element value of 0 represents the non-face area. In this way, the face area and non-face area of ​​the image can be determined by multiplying with the corresponding image.

[0068] The noise addition operation in step S55 can be implemented using a normal distribution algorithm, that is, , which can be based on the mean μ and variance Generate a noisy image. Among them, the variance The noise intensity at time step t is represented. During the noise addition process, the randomness of the noise addition process is controlled by increasing the noise intensity with time step t. The implementation process of this noise addition method is not described in detail in this application. The image degradation process from high quality to pure noise is simulated, and the intermediate image containing noise is obtained, i.e., the fused noisy image I t : ; (11) Based on this, in a possible implementation, the original user image and the target template image can be fused according to the dynamic weighting coefficient to obtain the fused image I mix : ; (12) Afterwards, the facial area of ​​the fused image can be denoised based on the dynamic facial mask, and the non-facial area of ​​the target template image can be gradually denoised. The noisy facial area and the noisy non-facial area can be fused to obtain the fused noisy image I t : ; (13) It can be seen that in the noise adding process, the facial area of ​​the original user image can be gradually noised, and the non-facial area of ​​the target template image can be noised, and then the noisy image areas of the two can be spliced ​​and fused to obtain a fused noisy image. The implementation process will not be described in detail in this application.

[0069] Step S56: Restoring the fused noisy image based on the dynamic weighting coefficient and the dynamic facial mask to obtain the target ID photo.

[0070] Integrating the operational principles of the diffusion algorithm, during the reverse denoising process, the fused noisy image, the first eigenvector, and the second eigenvector can be input into the denoising network in the image generation model. In one possible implementation, combined with the implementation method of the forward denoising process described above, the facial and non-facial regions in the fused noisy image can be progressively denoised based on the first eigenvector, the second eigenvector, and the dynamic weighting coefficient, respectively, to obtain the target ID photo. During the denoising process at each time step t, a feature-guided model can be used for denoising. This feature-guided denoising process can incorporate facial region features from the original user image and the target template image as conditional inputs, guiding noise removal as follows: ; (14) In the denoising process, the denoised image obtained at the previous time step can be expressed as: ; (15) In this way, the restoration weights of the facial and non-facial areas of the denoised image can be dynamically adjusted to ensure that the facial area features and light and shadow details can be reliably and completely preserved. After gradual denoising, the final target ID photo is generated at time step t=0. Among them, the different denoising treatments for the facial and non-facial areas can still be used. face Implementation, so formula (15) can be updated as: ; (16) Based on the above description of the image generation model's processing of input images, the training process of the image generation model can be implemented according to, but not limited to, the following methods: By acquiring real user images under various lighting conditions, with varying facial shapes, expressions, and skin tones, we ensure the adaptability of the image generation model to diverse inputs. Similarly, we can acquire standardized ID photos with varying lighting effects and background styles as initial sample template images. Afterwards, we can preprocess both the sample user images and the initial sample template images before using them for subsequent model training. Among them, during the preprocessing of the sample user image, geometric enhancement operations such as random rotation, scaling, or cropping can be performed to enhance the model's robustness to different postures, and by adjusting color attribute information such as brightness, contrast, and chroma to simulate the light changes in real shooting, artifacts and light and shadow changes can also be added to enhance the model's ability to learn light and shadow characteristics. This application does not limit the preprocessing method. The adjustment process for the initial sample template image is similar to the adjustment process of the original template image described in the above embodiment, and will not be repeated in this application.

[0071] Afterwards, a pre-trained model, such as a feature extraction network, can be used to extract the first sample feature vector of the sample user image and the second sample feature vector of the sample template image, respectively. Combined with the locations of facial key points, feature consistency and face shape matching can be optimized. After obtaining these contents, the sample user image and sample template image can be processed using the initial image generation model according to the method described above to obtain a predicted ID photo. Afterwards, a variety of loss functions can be used to calculate feature consistency loss, light and shadow consistency loss, adversarial loss, and reconstruction loss, respectively, to jointly optimize the quality of the generated predicted ID photo, ensuring the preservation of the user's facial features, the details of the template light and shadow, and the naturalness of the overall visual effect.

[0072] Optionally, the following formula can be used to obtain the feature consistency loss L based on the sample feature vectors of the sample fusion noise image, the sample user image, and the sample template image: feature : ; (17) Among them, F generated It can represent the sample feature vector of the predicted ID photo, and can represent the high-dimensional feature vector of the currently generated predicted ID photo on the facial area feature. The calculation method of the dynamic weighting coefficient λ is similar to the method described above and will not be described in detail in this application. By minimizing the feature consistency loss, F generated The weighted fusion result of the sample feature vector of the facial area of ​​the sample user image and the sample feature vector of the sample template image can be as close as possible.

[0073] Optionally, the present application can obtain the light and shadow consistency loss L according to the sample light and shadow intensity of the sample fusion noise image and the sample template image through the following formula: light ; ; (18) In formula (18), L generated The light and shadow residual of the generated predicted ID photo, i.e., the light and shadow distribution characteristics, can be extracted through image gradient analysis or a dedicated light and shadow feature extraction model to measure the brightness, shadow, and texture of the image. The implementation process is not described in detail in this application. At the same time, using this extraction method, the sample light and shadow residual L can be extracted from the sample template image. template , as a reference for standard light and shadow effects, the implementation process is not described in detail in this application. During the training process, this application hopes that L generated As close to L as possible template , to keep the lighting and shadow effects of the generated predicted ID photo consistent with the sample template image.

[0074] Optionally, based on the discriminator network, obtain the adversarial loss between the predicted ID photo output by the initial image generation model and the standard ID photo, calculated as follows: ; (19) Among them, D can represent the discriminator network, which is used to distinguish the real standard ID photo I target and the generated predicted ID photo I final The adversarial training module inputs the standard ID photo and the predicted ID photo for comparative analysis, and outputs the probability that the predicted ID photo is the real standard ID photo, that is, the prediction score. target Manually annotated or high-quality generated reference images can be used to represent the expected quality of the target ID photo. This application does not elaborate on the operating principle of the discriminator. During the training process, the realism of the predicted ID photo can be improved by maximizing the probability.

[0075] In addition, this application can also obtain the reconstruction loss L of the predicted ID photo based on the predicted ID photo and the standard ID photo output by the initial image generation model. recon , then, based on the feature consistency loss, light and shadow consistency loss, adversarial loss and reconstruction loss as well as the weights of each loss, the total predicted loss L is obtained total : ; (20) Among them, the above loss weight , β and The initial value of can be determined based on experience or historical data. =1, β=0.5, and =0.1, which is used to control the importance of feature consistency, light and shadow consistency, and adversarial training, so as to train the initial image generation model by minimizing the total prediction loss, that is, to adjust the parameters of the initial image generation model, such as the parameters of the diffusion model and the parameters of the discriminator network, so as to follow the preset learning rate (such as 10 -4 ) is iteratively trained until the total prediction loss converges or the preset number of iterations is reached to obtain an image generation model. This application does not elaborate on the implementation process of iterative training. The image generation model obtained according to the training method proposed in this application can reliably meet the diverse needs of users, improve universality, and ensure that its use in generating personalized ID photos strictly complies with the requirements of standard ID photos.

[0076] The above describes a method for generating an ID photo provided in an embodiment of the present application. The following describes a device for executing the above method for generating an ID photo.

[0077] Reference Figure 6 , is a structural diagram of a device for generating ID photos proposed in an embodiment of the present application, such as Figure 6 As shown, the ID photo generating device may include: An image acquisition module 61 is configured to acquire an original user image and an original template image corresponding to the original user image; the original template image is from a ID photo template library; A deformation processing module 62 is configured to perform image deformation on the original template image based on the original user image and an image deformation algorithm to obtain a target template image for the original user image; A generation module 63 is configured to input the original user image and the target template image into a trained image generation model, and process the corresponding areas between the original user image and the target template image according to different processing weights corresponding to facial areas and non-facial areas to obtain a target ID photo; Among them, the image deformation algorithm is obtained by constrained optimization of the thin plate spline TPS deformation algorithm, and the constraints include one or more of facial key point constraints, light and shadow consistency constraints, and facial structure constraints; the image generation model is obtained by optimizing and training the diffusion algorithm.

[0078] Those skilled in the art will understand that the functions and technical effects of each module in the above-mentioned device embodiment are equivalent to the corresponding steps described in the above-mentioned method embodiment. The specific implementation details can be referred to the description of the method part and will not be repeated here.

[0079] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the methods for generating ID photos provided in the embodiments of the present application.

[0080] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any one of the ID photo generation methods provided in the embodiment of the present application.

[0081] Reference Figure 7 , is a schematic diagram of the hardware structure of an electronic device proposed in an embodiment of the present application, such as Figure 7 As shown, the electronic device may include: at least one communication element 71, at least one memory 72 and at least one processor 73, wherein: The at least one communication element 71, the at least one memory 72 and the at least one processor 73 can communicate with each other via a bus. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The fact that only two lines are used does not mean that there is only one bus or one type of bus.

[0082] The communication element 71 may include a communication element that supports wireless communication methods such as Wi-Fi, Bluetooth, 5G / 6G and other mobile communications, so that the electronic device can transmit data with other devices through the communication element. It may also include one or more interfaces that support wired communication methods, such as a general-purpose input / output (GPIO) interface, a USB interface, a universal asynchronous receiver / transmitter (UART) interface, etc., to achieve data transmission between the various components within the electronic device. This application does not limit the composition structure of the communication element to achieve this function and its corresponding communication transmission mechanism, which can be determined according to the circumstances.

[0083] The memory 72 can be used to store multiple computer program instructions for implementing the ID photo generation method proposed in the embodiment of the present application; the processor 73 can load and execute the computer program instructions stored in the memory 72 to implement the various steps of the ID photo generation method proposed in the embodiment of the present application. The implementation process can refer to the description of the corresponding part of the method embodiment above.

[0084] In the embodiment of the present application, the memory 72 may include a storage medium such as a floppy disk, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. The processor 73 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA).

[0085] According to actual needs, the electronic device may also include at least one communication element for receiving the original time series data sent by the acquisition device or sending it to other devices (such as servers, etc.) for processing. It can also be used to realize data or instruction transmission between internal components of the electronic device.

[0086] It should be understood that Figure 7 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 7 More or fewer components shown, or combinations of certain components, when the electronic device is a terminal device such as a mobile phone, tablet computer, wearable device, vehicle-mounted device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultramobile personal computer (UMPC), netbook, smart transportation / medical equipment, smart home device or robot, may also include microphones, speakers, displays, various sensors, antennas, power modules, radio frequency components, external ports and other input / output components, etc., which can be determined based on processing function requirements, and this application does not provide detailed examples one by one.

[0087] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0088] Through the description of the above embodiments, in the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training equipment or data center to another website, computer, training equipment or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, data center, etc. that includes one or more available media integrations. The available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)).

[0089] In addition, the various embodiments in this specification are described in a progressive or parallel manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. The devices, electronic devices, products, and media disclosed in the embodiments are described briefly because they correspond to the methods disclosed in the embodiments. For relevant details, refer to the method description.

Claims

1. A method for generating an ID photo, characterized in that: The method comprises: Obtaining an original user image and an original template image corresponding to the original user image; the original template image is from a ID photo template library; Performing image deformation on the original template image according to the original user image and an image deformation algorithm to obtain a target template image for the original user image; Inputting the original user image and the target template image into a trained image generation model, processing the corresponding areas between the original user image and the target template image according to different processing weights corresponding to facial areas and non-facial areas, to obtain a target ID photo; The image deformation algorithm is obtained by constraining optimization of the thin plate spline TPS deformation algorithm, wherein the constraints include one or more of facial key point constraints, light and shadow consistency constraints, and facial structure constraints; The image generation model is obtained by optimizing and training the diffusion algorithm.

2. The method according to claim 1, characterized in that The step of performing image deformation on the original template image based on the original user image and the image deformation algorithm to obtain a target template image for the original user image includes: Performing facial key point detection on the original template image to obtain a plurality of original facial key point positions; and performing facial key point detection on the original user image to obtain a plurality of user facial key point positions; Dynamically matching the user's facial key point positions to obtain multiple target facial key point positions; According to the multiple original facial key point positions and the multiple target facial key point positions, nonlinearly deform the facial key area of ​​the original template image through an image deformation algorithm, and smoothly transition the facial non-key area of ​​the original template image, and obtain a deformed template image that satisfies the facial key point constraints through bilinear interpolation calculation; The deformed template image is processed according to at least one of the light and shadow consistency constraint and the facial structure constraint to obtain a target template image for the original user image.

3. The method according to claim 2, characterized in that The method comprises performing nonlinear deformation on the facial key area of ​​the original template image and performing smooth transition on the facial non-key area of ​​the original template image by using an image deformation algorithm based on the multiple original facial key point positions and the multiple target facial key point positions, including: Obtaining deformation weights corresponding to different facial key areas and non-key areas; According to the deformation weights, nonlinear deformation of the facial key area of ​​the original template image and smooth transition of the facial non-key area are achieved by mapping the multiple original facial key point positions to the target facial key point positions of the corresponding facial key points, so as to minimize the distance between the original facial key point positions and the target facial key point positions corresponding to the same facial key point; Among them, the deformation weight corresponding to the facial key point is greater than the deformation weight corresponding to the facial non-key point; the deformation weight corresponding to the facial key point that has a greater impact on the quality of the ID photo is greater.

4. The method according to claim 2, characterized in that Processing the deformed template image according to the light and shadow consistency constraint includes: Acquire a first light and shadow intensity of the original template image and a second light and shadow intensity of the deformed template image; Obtaining a light and shadow residual between the original template image and the deformed template image according to a difference between the first light and shadow intensity and the second light and shadow intensity at the same pixel point; The deformed template image is smoothed according to the light and shadow residual and the light and shadow constraint weight, so that the light and shadow effect of the processed template image is consistent with the light and shadow effect of the original template image.

5. The method according to claim 2, characterized in that Processing the deformed template image according to the facial structure constraint includes: determining a facial symmetry line in the deformed template image; According to the facial symmetry line and the symmetry constraint weight, the deformed template image is subjected to point-wise mirror transformation so that the facial key points in the transformed template image are bilaterally symmetrical about the facial symmetry line.

6. The method according to any one of claims 1 to 5, characterized in that The step of inputting the original user image and the target template image into a trained image generation model, processing corresponding areas between the original user image and the target template image according to different processing weights corresponding to facial areas and non-facial areas, and obtaining a target ID photo, includes: Inputting the original user image and the target template image into a trained image fusion model, fusing and denoising the facial regions and non-facial regions of the original user image and the target template image respectively according to the fusion weight and the dynamic facial mask, to obtain a fused noisy image; The facial region and the non-facial region of the fused and noisy image are restored respectively according to the generated weights and the dynamic facial mask to obtain a target ID photo.

7. The method according to claim 6, characterized in that The fusion weight and the generated weight are the same dynamic weighting coefficient, and the method for obtaining the dynamic weighting coefficient includes: Performing feature extraction on the facial region of the original user image to obtain a first feature vector, and performing feature extraction on the target template image to obtain a second feature vector; performing similarity calculation on the first eigenvector and the second eigenvector to obtain image similarity; Matching the facial key point positions between the original user image and the target template image to obtain a facial key point matching degree; The image similarity and the facial key point matching degree are weighted and summed, and normalized to obtain dynamic weighting coefficients for the original user image and the target template image.

8. The method according to claim 7, characterized in that The matching of facial key point positions between the original user image and the target template image to obtain a facial key point matching degree includes: Aligning the facial center points of the original user image and the target template image; The relative average offset between the aligned original user image and the target template image is calculated to obtain the facial key point matching degree between the original user image and the target template image.

9. The method according to claim 7, characterized in that The method comprises: fusing and denoising the facial region and non-facial region of the original user image and the target template image based on the fusion weight and the dynamic facial mask to obtain a fused noisy image; and restoring the facial region and non-facial region of the fused noisy image based on the generated weight and the dynamic facial mask to obtain a target ID photo. fusing the original user image and the target template image according to the dynamic weighting coefficient to obtain a fused image; Noising the facial region of the fused image and gradually noising the non-facial region of the target template image based on the dynamic facial mask, fusing the noisy facial region with the noisy non-facial region to obtain a fused noisy image; Based on the first eigenvector, the second eigenvector and the dynamic weighting coefficient, the facial area and the non-facial area in the fused noisy image are gradually denoised to obtain a target ID photo.

10. The method according to claim 9, characterized in that In the process of obtaining the image generation model by optimizing and training the diffusion algorithm: Obtain feature consistency loss based on the sample feature vectors of the sample fusion noisy image, the sample user image, and the sample template image; Obtain the light and shadow consistency loss based on the sample light and shadow intensities of the sample fusion noisy image and the sample template image; Based on the discriminator network, obtain the adversarial loss between the predicted ID photo output by the initial image generation model and the standard ID photo; Obtaining a reconstruction loss of the predicted ID photo based on the predicted ID photo and the standard ID photo output by the initial image generation model; Obtaining a total prediction loss according to the feature consistency loss, the light and shadow consistency loss, the adversarial loss, the reconstruction loss, and the weights of each loss; The initial image generation model is trained by minimizing the total prediction loss to obtain an image generation model.

11. An electronic device, characterized in that: The electronic device comprises: at least one communication element, at least one memory and at least one processor, wherein: The communication element is configured to receive an original user image and an original template image corresponding to the original user image; the original template image is from a ID photo template library; The memory is used to store a plurality of computer program instructions; The processor is configured to load and execute the computer program instructions to implement the following steps: Performing image deformation on the original template image according to the original user image and an image deformation algorithm to obtain a target template image for the original user image; Inputting the original user image and the target template image into a trained image generation model, processing the corresponding areas between the original user image and the target template image according to different processing weights corresponding to facial areas and non-facial areas, to obtain a target ID photo; The image deformation algorithm is obtained by constraining optimization of the thin plate spline TPS deformation algorithm, wherein the constraints include one or more of facial key point constraints, light and shadow consistency constraints, and facial structure constraints; The image generation model is obtained by optimizing and training the diffusion algorithm.

Images