Personalized customization drawing method and system

Abstract

One or more embodiments of the invention provide a personalized customized drawing method and system. The method comprises the following steps: determining a face description text specified by a user, and calling a generative algorithm to generate a face image based on the face description text; obtaining a first dressing effect image of a target garment, wherein the first dressing effect image is used for presenting a wearing effect of the target garment on the digital model; and performing fusion processing on the face image and the first dressing effect image to generate a second dressing effect image of the target garment, the second dressing effect image being used for presenting a wearing effect of the target garment on the fused digital model. According to the scheme, the garment picture can be quickly output with high quality, the fused image can keep feature consistency, and the display effect of the image can also meet the personalized requirements of users.

Description

Technical Field

[0001] One or more embodiments of the present invention relate to the field of terminal technology, and in particular, to a personalized customized image output method and system. Background Art

[0002] With the popularity of online shopping and the increase in personalized consumption demand, people have put forward higher requirements for personalized customization of clothing. Traditional clothing customization methods mainly rely on graphic design and fitting, which cannot intuitively show the effect of clothing on models, and it is difficult to intuitively display the actual effect of clothing on different human bodies.

[0003] At the same time, in the field of clothing customization and design, it is of great significance to provide customers with personalized clothing effect displays. Users often hope to be able to adjust clothing styles, colors, accessories, etc. according to their own needs and preferences, and be able to preview the finished product effects through various elements of the model.

[0004] There are some methods for customizing images in the prior art, such as manual drawing, image editing software, or virtual reality technology, etc. However, these methods still have some limitations and shortcomings, such as lack of realism and accuracy, etc. In addition, customizing images requires a lot of manpower and time, and using image editing software to produce images requires users to have certain image processing skills, while virtual reality technology requires less expensive equipment and professional knowledge, and is more complicated for ordinary users to operate. Summary of the invention

[0005] In view of this, one or more embodiments of the present invention provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of the present invention, a personalized customized drawing method is proposed, comprising:

[0007] Determine a face description text specified by a user, and call a generative algorithm to generate a face image based on the face description text;

[0008] Acquire a first dressing effect image of the target garment, where the first dressing effect image is used to present the wearing effect of the target garment on a digital model;

[0009] The face image and the first dressing effect image are fused to generate a second dressing effect image of the target garment, wherein the second dressing effect image is used to present the wearing effect of the target garment on the fused digital model.

[0010] According to a second aspect of one or more embodiments of the present invention, a personalized customized image output system is provided, comprising:

[0011] A face image generation module, used to determine a face description text specified by a user, and to call a generative algorithm to generate a face image based on the face description text;

[0012] A dressing image acquisition module, used for acquiring a first dressing effect image of a target garment, wherein the first dressing effect image is used for presenting a wearing effect of the target garment on a digital model;

[0013] An image fusion module is used to fuse the face image and the first dressing effect image to generate a second dressing effect image of the target clothing, wherein the second dressing effect image is used to present the wearing effect of the target clothing on the fused digital model.

[0014] According to a third aspect of one or more embodiments of the present invention, a method for personalized customized drawing is proposed, the method comprising:

[0015] Performing three-dimensional modeling on the clothing and the model to generate a three-dimensional model, and rendering the three-dimensional model to obtain a rendered image;

[0016] Analyzing clothing elements and model elements of the rendered image;

[0017] The rendered image is modified by adjusting the clothing elements, the model elements and / or adding new elements to obtain a finished image.

[0018] According to a fourth aspect of one or more embodiments of the present invention, a personalized customized image output system is provided, the system comprising:

[0019] A modeling module, used for three-dimensional modeling of clothing and models to generate a three-dimensional model;

[0020] A rendering module, used for rendering the three-dimensional model to obtain a rendered image;

[0021] A parsing module, used for parsing clothing elements and model elements of the rendered image;

[0022] The modification module is used to modify the rendered image by adjusting the clothing elements, the model elements and / or adding new elements to obtain a finished image.

[0023] According to a fifth aspect of one or more embodiments of the present invention, a model face customization method based on maintaining facial consistency is proposed, comprising:

[0024] Acquire a source model face, and extract facial data and / or hair data of the source model face;

[0025] Directly obtain a reference model face; or use a preset face parameterization customization module to perform parameterization description from one or more dimensions of basic features, facial features, and hair features to obtain a reference model face;

[0026] Extracting a first feature vector from the facial data and / or the hair data of the source model face, and performing face recognition on the reference model face to extract a second feature vector;

[0027] The first feature vector and the second feature vector are subjected to feature fusion to obtain a latent space representation; the latent space representation is decoded to obtain a target model face.

[0028] According to a sixth aspect of one or more embodiments of the present invention, a model face customization system based on maintaining facial consistency is proposed, comprising:

[0029] A first acquisition unit, used to acquire a source model face, and extract facial data and / or hair data of the source model face;

[0030] The second acquisition unit directly acquires the reference model face; or uses a preset face parameterization customization module to perform parameterization description from one or more dimensions of basic features, facial features and hair features to obtain the reference model face;

[0031] A feature vector unit, extracting a first feature vector from the facial data of the source model face and / or the hair data, and performing face recognition on the reference model face to extract a second feature vector;

[0032] A feature fusion unit, which performs feature fusion on the first feature vector and the second feature vector to obtain a latent space representation;

[0033] The generation unit is used to decode the latent space representation to obtain a target model face, and use a face beautification module to optimize and improve the recognizability and expressiveness of the target model face.

[0034] According to a seventh aspect of one or more embodiments of the present invention, an electronic device is proposed, comprising: a processor; a memory for storing processor executable instructions; wherein the processor implements the steps of the method described in the first aspect by running the executable instructions.

[0035] According to an eighth aspect of one or more embodiments of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0036] According to a ninth aspect of one or more embodiments of the present invention, a computer program product is proposed, comprising a computer program / instruction, which implements the steps of the method described in the first aspect when executed by a processor.

[0037] Among them, the above solution of the present invention can achieve the following technical effects:

[0038] By using the personalized customized drawing method in the first aspect, it is possible to quickly draw clothing pictures with high quality, improve the efficiency of drawing and content distribution and marketing needs, maintain the high fidelity of modeled clothing, and the operation is not difficult, which significantly reduces the time cost and labor cost, and also has a wider range of applications. In addition, the user can specify the face description text according to their own needs to call the generative algorithm to generate a face image that meets the needs based on the text. Compared with the face image drawn by the user or intercepted from the 3D model, the face image generated by this method is more realistic (that is, the CG sense is weaker), which helps to ensure that the second dressing effect image obtained by the fusion process has a strong sense of authenticity. In addition, this solution fuses the face image and the first dressing effect image for presenting the wearing effect of the target clothing on the digital model to obtain a second dressing effect image for presenting the wearing effect of the target clothing on the fused digital model. This image can maintain the consistency of the features of the digital model in the face image and the first dressing effect image, that is, the appearance of the fused digital model in the image can present both the facial features in the above-mentioned face image and the original human features of the digital model. The display effect of the second dressing effect image can also meet the above-mentioned needs of the user. Obviously, compared with the related technical solutions of directly cutting, splicing / replacing faces, the human body display effect in the second dressing effect image obtained by the fusion processing of this solution is more realistic and natural.

[0039] By utilizing the personalized customized rendering method in the third aspect mentioned above, it is possible to quickly render clothing images with high quality, improve the rendering and content distribution efficiency and marketing needs, maintain high fidelity of modeled clothing, and have low operating difficulty, significantly reducing time and labor costs, while also having a relatively wide range of applications.

[0040] By using the model face customization method based on maintaining facial consistency in the fifth aspect, a target model face can be quickly output given a source model face and a reference model face, and the feature consistency of the source model face and the reference model face can be effectively maintained, and the personalized features on the model face can be retained, so as to achieve high-quality output of the model face, and the model face details with high image resolution and rich details can be accurately modified and customized. The face parameterized customization module can meet the highly personalized customization of the reference model face, thereby realizing the application of more demand scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of the overall module structure of the personalized customized drawing system provided in Example 1.

[0042] Figure 2 A schematic diagram of the overall process of the personalized customized drawing method provided in Example 2.

[0043] Figure 3 Schematic diagram of the process of generating a three-dimensional model.

[0044] Figure 4 Schematic diagram of the process to obtain the finished product picture.

[0045] Figure 5 This is a flow chart of the model face customization method provided in Example 3.

[0046] Figure 6 This is a schematic diagram of the principle of the model face customization method provided in Example 3.

[0047] Figure 7 A schematic diagram of a feature fusion step provided in an embodiment of the present invention.

[0048] Figure 8 This is a schematic diagram of the structure of the model face customization system provided in Example 4.

[0049] Fig. 9 This is a schematic diagram of the overall process of the personalized customized drawing method provided in Example 5.

[0050] Fig.10 A schematic diagram of the personalized drawing operation entry interface.

[0051] Fig.11 A schematic diagram of the personalized drawing operation interface.

[0052] Fig.12a A schematic diagram of the selection interface for face description options.

[0053] Figure 12b A schematic diagram of a face fusion processing result.

[0054] Fig.13a Schematic diagram of the input interface for face description text.

[0055] Fig.13b A schematic diagram of another face fusion processing result.

[0056] Fig.14 A schematic diagram of the overall module structure of the personalized customized drawing system provided in Example 6.

[0057] Fig.15A schematic structural diagram of a device is provided for an exemplary embodiment. DETAILED DESCRIPTION

[0058] In order to solve the above-mentioned problems existing in the related art, the present invention proposes a personalized customized image output method, which allows the user to flexibly adjust the elements in the image to modify the rendered image. In addition, the present invention also proposes a model face customization method based on facial consistency maintenance, that is, first extracting the first feature vector of the source model face and the second feature vector of the reference model face, and then performing feature fusion on the two feature vectors to obtain the target model face. On this basis, the present invention proposes another personalized customized image output method based on image fusion, that is, determining the face image according to the face image uploaded by the user or according to the face description information specified by the user, and then fusing the face image with the first dressing effect image to obtain the second dressing effect image. The following is a detailed description of the three methods and their corresponding systems in conjunction with the accompanying drawings and related embodiments.

[0059] In the following, in combination with Example 1 and Example 2, a personalized customized drawing system and method proposed by the present invention are respectively described.

[0060] [Example 1]

[0061] This embodiment proposes a personalized customized image output system, which can realize the rapid output of three-dimensional clothing models with high quality, and can realize the efficient single image and group image output of any scene, model, posture, and perspective, improve the content distribution efficiency and marketing needs, while maintaining the high fidelity of the modeled clothing. Figure 1 , the system comprising:

[0062] Modeling module 1, used for performing three-dimensional modeling of clothing and models to generate a three-dimensional model;

[0063] Rendering module 2, used for rendering the three-dimensional model to obtain a rendered image;

[0064] Parsing module 3, used to parse clothing elements and model elements of the rendered image;

[0065] The modification module 4 is used to modify the rendered image by adjusting clothing elements, model elements and / or adding new elements to obtain a finished image.

[0066] In this embodiment, the rendering module 2 is used to render the clothing in the three-dimensional model to make the clothing more realistic; the clothing elements analyzed by the analysis module 3 and adjusted by the modification module 4 may include the outline, texture, color and / or style of the clothing; the model elements may include the model's skin color, expression, movement, face shape, hairstyle, figure and / or makeup, etc.

[0067] In one embodiment, the modification module 4 also supports the prior information parsed by the parsing module 3 from clothing, models, etc., and uses this prior information as controllable and stably generated advance knowledge of the generative AI, effectively ensuring the robustness of the system operation.

[0068] It should be noted that the pictures obtained by the personalized customized image output system proposed in this embodiment can be a single picture or a set of multiple pictures, which fully utilizes the digital modeling capabilities and does not require highly complex model modeling. At the same time, there is no need to go through traditional sample clothing, model shooting, photo editing and other operations, thereby achieving rapid image output and content distribution.

[0069] For example, when a user needs to obtain a set of multiple pictures, the clothing elements, model elements and newly added elements can be adjusted accordingly, so that some of the elements are the same, while other elements can be changed according to the user's needs, and finally the set of pictures required by the user is generated.

[0070] For example, the models in the generated set of pictures can be fixed to the same model by making some model elements the same, while changing other model elements to make the models have different expressions, movements, and postures. The clothes in the generated set of pictures can be fixed to the same style by making some clothing elements the same, while changing other clothing elements to make the clothes have different colors and textures. The clothes in the generated set of pictures can be fixed to the same background by making some scene elements the same, while changing other scene elements to make the scenes have different perspectives.

[0071] In this embodiment, the modeling module 1 can complete the three-dimensional modeling of clothing and models in clothing modeling software such as Style3D Studio. For example, 3D clothing modeling is first performed, and after the clothing modeling is completed, the user can select a digital model or a mannequin model to generate a three-dimensional model.

[0072] In one embodiment, the modeling module 1 may include: a clothing modeling unit 11, which is used to perform three-dimensional modeling of clothing to construct a three-dimensional clothing model; a model modeling unit 12, which is used to provide users with the option of customizing digital models, and based on the three-dimensional clothing model and the customized digital model, perform clothing simulation in the modeling software to generate a three-dimensional model; or provide users with multiple mannequins for users to choose from, and simulate the three-dimensional clothing model and the mannequin selected by the user to a stable state through the simulation engine of the modeling software to generate a three-dimensional model.

[0073] In one embodiment, the modeling module 1 can also model the scene while modeling the clothing and the model, and can also add scene elements through the modification module 4 to automatically generate a picture scene.

[0074] When the modeling module 1 models the scene while modeling the clothing and the model, the modeling module 1 may also include a scene modeling unit 13, which is used to preset the scene suitable for the clothing and the model when generating the three-dimensional model, so as to generate a three-dimensional model including the scene. Accordingly, the parsing module 3 may include: a clothing parsing unit 31, which is used to parse the clothing elements of the rendered image; a model parsing unit 32, which is used to parse the model elements of the rendered image; and a scene parsing unit 33, which is used to parse the scene elements of the rendered image.

[0075] In this way, the presentation effects of clothing on different models can be achieved, such as hairstyle, face shape, expression, posture, etc., which can show the effect of clothing more vividly and realistically; at the same time, the presentation scenes of clothing can be customized, such as beaches, street shots, catwalks, etc., which is more helpful to enhance the different concerns of different groups on clothing performance.

[0076] In one embodiment, the modification module 4 may include: an encoder 41, used to encode the rendered image; an adjustment unit 42, used to modify the encoded rendered image by adjusting clothing elements, model elements and / or adding new elements to obtain a modified image; a decoder 43, used to decode the modified image to obtain a finished image.

[0077] It should be noted that the encoder 41 can convert network calculations from the image pixel space to the latent space, achieve data dimensionality reduction and compression, and reduce the amount of calculation. This can improve the generation quality of the model, better control the input conditions of the adjustment unit 42, and can be applied to some occasions that require special viewing angles, thereby better meeting design requirements and improving efficiency.

[0078] The elements added by the adjustment unit 42 may include but are not limited to:

[0079] Scene elements, so that the adjustment unit 42 modifies the scene of the rendered image based on the scene elements, and the scene elements include but are not limited to the color tone, layout, matching and other elements of the scene;

[0080] Character elements, so that the adjustment unit 42 modifies the model based on the character elements, the character elements including but not limited to the model's age, gender, action, etc.;

[0081] Text information, so that the adjustment unit 42 displays the text information on the rendered image. The text information may include clothing prices, current hot items, fashion trends, current affairs and other information closely related to business, so that the personalized customized drawing system proposed in this embodiment can be applied to the rapid customized drawing of online e-commerce platforms.

[0082] By quickly producing images from digital modeling to e-commerce scenarios, we can fully utilize the digital modeling capabilities without the need for highly complex modeling. At the same time, there is no need to go through traditional operations such as sampling, model shooting, and photo editing. We can achieve rapid e-commerce image production and rapid content distribution with one click.

[0083] In one embodiment, the adjustment unit 42 may include a generation network. It should be noted that the generation network is a machine model with a learning function. The generation network can extract rules and patterns from existing data through learning based on probability models and statistical methods, and use these patterns to generate new data, such as generating new samples of images, texts, audio and other data.

[0084] [Example 2]

[0085] Corresponding to the personalized customized image output system described in the above embodiment 1, this embodiment proposes a personalized customized image output method. Figure 2 , the method comprising:

[0086] S210: Perform three-dimensional modeling on the clothing and the model to generate a three-dimensional model, and render the three-dimensional model to obtain a rendered image.

[0087] The rendering of the three-dimensional model in S210 may specifically be rendering the clothing in the three-dimensional model to make the clothing more realistic.

[0088] S220: Analyze clothing elements and model elements of the rendered image.

[0089] In this embodiment, the model elements may include the model's skin color, expression, action, face shape, hairstyle, figure and / or makeup; the clothing elements may include the outline, texture, color and / or style of the clothing.

[0090] S230: Modify the rendered image by adjusting clothing elements, model elements and / or adding new elements to obtain a finished image.

[0091] It should be noted that the finished image obtained in step S230 can be a single image or a set of multiple images. Through this method, the digital modeling capability can be fully utilized without the need for highly complex modeling of models, and at the same time, there is no need to go through traditional operations such as sample clothing, model shooting, and photo editing, thereby achieving rapid image output and content distribution.

[0092] When the user needs to obtain a set of multiple groups of pictures, the clothing elements, model elements and newly added elements can be adjusted accordingly, so that some of the elements are the same, while other elements can be changed according to the user's needs, and finally the group of pictures required by the user is generated.

[0093] Exemplarily, the models in the generated set of pictures can be fixed to the same by making some of the model elements the same, while changing other model elements so that the models have different expressions, movements, and postures; the clothes in the generated set of pictures can be fixed to the same style by making some of the clothing elements the same, while changing other clothing elements so that the clothes have different colors and textures; the clothes in the generated set of pictures can be fixed to the same background by making some of the scene elements the same, while changing other scene elements so that the scenes have different perspectives.

[0094] In one embodiment, in step S210, the three-dimensional modeling of the clothing and the model can be completed in clothing modeling software such as Style3D Studio. For example, 3D clothing modeling is first performed, and after the clothing modeling is completed, the user can select a digital model or a mannequin model to generate a three-dimensional model. Figure 3 The process of performing three-dimensional modeling on the clothing and the model to generate a three-dimensional model in step S210 may include:

[0095] S211: Perform three-dimensional modeling on the clothing to construct a three-dimensional clothing model.

[0096] After step S211, the user can choose to execute one of step S212 and step S213.

[0097] S212: providing the user with an option to customize a digital model, and performing clothing simulation in a modeling software to generate a three-dimensional model based on the three-dimensional clothing model and the customized digital model.

[0098] S213: providing a plurality of mannequins for the user to choose from, simulating the three-dimensional clothing model and the mannequin selected by the user to a stable state through a simulation engine of a modeling software to generate a three-dimensional model.

[0099] In one embodiment, when generating a three-dimensional model in step S210, a scene suitable for the clothing and the model can be preset to generate a three-dimensional model including the scene. When parsing the clothing elements and model elements of the rendered image in step S220, the scene elements of the rendered image are also parsed, so that the rendered image can be modified by adjusting the scene elements in step S230. In addition, the picture scene can also be generated by directly adding scene elements in step S230.

[0100] See also Figure 4 , step S230 may include:

[0101] S231: Encode the rendered image.

[0102] S232: Modify the encoded rendering image by adjusting clothing elements, model elements and / or adding new elements to obtain a modified image.

[0103] S233: Decode the modified image to obtain a finished image.

[0104] Among them, step S231 can be implemented by an encoder, which can convert network calculations from pixel space to latent space, achieve data dimensionality reduction and compression, and reduce the amount of calculation. This can improve the generation quality of the model, better adjust or add various elements, and can be applied to some occasions that require special viewing angles, thereby better meeting design requirements and improving efficiency.

[0105] In addition, the modification of the rendered image by adding new elements as described in step S232 may include: adding scene elements to modify the scene of the rendered image based on the scene elements, and the scene elements include but are not limited to the color tone, layout, matching and other elements of the scene; and / or, adding character elements to modify the model based on the character elements, and the character elements include but are not limited to the model's age, gender, action and other elements; and / or, adding text information to display the text information on the rendered image, and the text information may include clothing prices, current hot items, fashion trends, current affairs and other information closely related to business.

[0106] In this embodiment, step S232 can be implemented by a generative network. It should be noted that the generative network is a machine model with learning capabilities. The generative network can extract rules and patterns from existing data through learning based on probability models and statistical methods, and use these patterns to generate new data, such as generating new samples of images, texts, audio and other data.

[0107] The solution proposed in this embodiment has a wide range of applications. For example, when it is applied to the rapid customization of images on online e-commerce platforms, by processing and producing materials such as three-dimensional digital modeling clothing, models, scenes, etc., it can meet the generative AI technology steps required for content distribution in e-commerce scenarios, and can use this step to achieve the purpose of rapid and high-quality personalized customization of single images and group images for e-commerce models.

[0108] As can be seen from the above-mentioned Embodiments 1 and 2, the personalized customized image rendering system provided by the present invention has the function of analyzing multiple elements, so that users can obtain more realistic and lifelike images according to their needs and preferences. Users can customize the rendered images by adjusting clothing elements, model elements, or adding new elements to accurately present the effect of clothing on the model.

[0109] Furthermore, this solution can achieve the presentation effect of clothing on different models, such as hairstyle, face shape, expression, posture, etc., to show the effect of clothing more vividly and realistically; at the same time, the presentation scene of clothing can be customized, such as beach, street shooting, show, etc., which is more helpful to enhance the different concerns of different groups on clothing performance;

[0110] Preferably, this solution can be applied to the rapid customization of graphics on online e-commerce platforms. By rapidly producing graphics from digital modeling to e-commerce scenarios, the digital modeling capabilities can be fully utilized without the need for highly complex modeling of models. At the same time, there is no need to go through traditional operations such as sample clothing, model shooting, and photo editing. Rapid e-commerce graphics and content distribution can be achieved with one click.

[0111] In summary, the personalized customized drawing system and method provided by the present invention can realize the rapid drawing of clothing pictures with high quality, improve the drawing and content distribution efficiency and marketing needs, maintain the high fidelity of modeled clothing, have low operating difficulty, significantly reduce time and labor costs, and also have a relatively wide range of applications.

[0112] In addition, the present invention also proposes a model face customization method and system based on maintaining facial consistency. The method and system are described below in conjunction with Embodiment 3 and Embodiment 4.

[0113] [Example 3]

[0114] This embodiment proposes a model face customization method based on maintaining facial consistency. This method can effectively maintain the feature consistency of the source model face and the reference model face, achieve high-quality model face output, and meet the requirements of highly personalized customization of the reference model face. The specific process of this method is as shown in the attached manual. Figure 5 As shown, the method includes:

[0115] S510: Acquire a source model face, and extract facial data and / or hair data of the source model face;

[0116] S520: directly obtaining a reference model face; or using a preset face parameterization customization module to perform parameterization description from one or more dimensions of basic features, facial features, and hair features to obtain a reference model face;

[0117] S530: extracting a first feature vector from the facial data and / or hair data of the source model face, and performing face recognition on the reference model face to extract a second feature vector;

[0118] S540: performing feature fusion on the first eigenvector and the second eigenvector to obtain a latent space representation;

[0119] S550: Decode the latent space representation to obtain the target model face.

[0120] In one embodiment, after step S550, the method further includes:

[0121] S560: Use the preset face beautification module to optimize and improve the recognizability and expressiveness of the target model's face.

[0122] In this embodiment, there are a source model face and a reference model face, which can be understood as fusing the identity features of the reference model face with the attribute features of the source model face to keep the facial feature migration consistent. However, it should be noted that the above fusion is not a simple face replacement, but will retain the personalized and significant features of the source model face and the reference model face. The features on the model face finally generated are all fused to a certain extent, and are not just derived from one face; in other words, the target model face generated by the fusion is a new model that has the significant features of both the source model face and the reference model face.

[0123] Among them, attribute features refer to some inherent features of the human face, which can include: Structural features: The human face has a complex structure, such as the position, size, shape, etc. of the eyes, mouth, nose, eyebrows, ears and other parts. Facial expressions: The human face can express a variety of emotions and feelings, such as joy, anger, sorrow, and happiness. Skin color and texture: Different races may have different skin colors and facial textures, which is also an attribute feature of the human face. Other features: The human face has other attribute features, such as face shape, facial contour, etc.

[0124] Among them, identity features refer to each person's unique individual features. Specifically, they include: 3D features of the face: This refers to the overall shape of the face, including the relative position and size of the facial organs. Color and texture features of the face: This refers to the skin color, skin color, facial texture, etc. Micro-expression features of the face: This refers to the subtle expressions of the face in a short period of time. These expressions are usually unconscious, but can be identified and analyzed through machine learning and image processing technology.

[0125] Among them, the significant features can be understood as the personalized features of the model's face, such as a unique hairstyle, a unique makeup, etc., because the presence of significant features makes the model have a sense of luxury. There is no standard for the sense of luxury itself, and there is no specific quantitative means for significant features. Therefore, significant features can be selected artificially, and any features that the user feels like customizing on the model's face can be used as significant features. And the sense of luxury is a manifestation of its unique characteristics for the model, which is also emphasized by the scheme of this embodiment. The scheme of this embodiment can achieve the customization of the model's face while retaining the unique significant features on the model's face, so that the generated target model face has a sense of luxury. Preferably, the features with personality on the source model face and the reference model face are selected as significant features, and the significant features include the identity features of the source model face and the identity features of the reference model face; the latent space representation generated later includes significant features, features after the fusion of the significant features of the source model face and the reference model face, and features after the fusion of the non-significant features of the source model face and the reference model face, so that the finally generated customized face has the personalized features on the source model face and the reference model face, and a model face with a sense of luxury is obtained.

[0126] Specifically, after a source model face is given, it is necessary to perform facial analysis on it to effectively locate the model face and extract the facial data and / or hair data of the source model. Algorithms such as facial key points and face segmentation can be used, such as Figure 6 As shown. In practical applications, multiple algorithms may be combined to obtain the best face detection effect. For example, a rectangular area is demarcated on the source model's face, the location of the area and the features within the area are determined, or the mask area data of the source model's face is obtained. Facial data at least includes facial contours, the type of face, and various details of the face. This embodiment uses a face analysis algorithm to divide the face into regions, and it is necessary to determine the location, type, size, and features of the region. Hair data mainly refers to the location, color, shape, etc. of the hair in the source model's face, and data related to the hair can be included in the hair data. Subsequent feature transplantation is performed in the face area and / or hair area, so it is necessary to obtain relevant information about the area to facilitate subsequent face transplantation.

[0127] The reference model face can be understood as the target of the target model face. There are two ways to input the reference model face. One is to directly input the model face. For example, the user directly inputs the image of the reference model face that meets the conditions into the software, or generates the corresponding image after the data of the reference model face is given; the other is to use the face parameterization customization module to obtain the image of a specific reference model face. Among them, the face parameterization customization module is different from the traditional face customization solution. It not only systematizes and details the characteristic attributes of each dimension, but also improves the confusion problem caused by the coupling relationship between the attributes in the description.

[0128] In this embodiment, the face parameter customization module describes the customized model from three dimensions: basic facial features, facial features, and hair features. Figure 6 As shown. Basic features involve basic information that affects the face, including gender, age, expression, skin color, face shape, makeup, facial details, facial contours and model style. Facial details mainly refer to detailed features such as wrinkles on the face. Model style will affect the subsequent feature description coupling problem. Model faces of different styles may correspond to different feature descriptions. Facial features include eyebrows, mouth, eyes, nose, and ears. Detailed features such as eyebrows, lips, eyes, pupil color, nose, and chin are all customizable. Hair features include hairstyle, hair color, and hair length. The global features of the model face are customized from three dimensions: basic facial features, facial features, and hair features. At the same time, the detailed features under each dimension are specifically refined, so that fine-grained detailed features can be uniquely designed to create a unique model face expression.

[0129] The face parameter customization module forms a parametric description of the model's face based on the set basic features, facial features, and hair features. The description can be set based on a specific reference model face image or based on application experience to meet the customization requirements for the model's face. The face parameter customization module can meet the highly personalized customization of the reference model's face, thereby realizing the application of more demand scenarios. For the parametric description, generative algorithms such as generative networks, such as GAN and diffusion models, are used to generate model faces that meet the feature requirements, effectively achieving the uniqueness and aesthetic visual effects of the model's face.

[0130] In a specific embodiment, in the face parameterization customization module, each characteristic attribute on the reference model's face is described by a descriptive word to achieve a parametric description; and a token processing mechanism is pre-built to handle the confusion of characteristic attribute descriptions caused by the coupling of descriptive words between characteristic attributes. The token processing mechanism associates facial features with the general description of the feature, and avoids the confusion caused by the coupling between characteristic descriptions. The token mechanism establishes the difference between adjacent descriptive words and non-adjacent descriptive words, so that adjacent descriptive words have higher weights and non-adjacent descriptive words have lower weights, which is similar to the intra-class spacing and inter-class spacing of clustering problems. For example, red eyebrows and light lips, the token processing mechanism associates red with eyebrows and light with lips, and avoids the confusion caused by the coupling between characteristic descriptions such as red lips and light eyebrows.

[0131] Further preferably, a facial feature description mechanism with general rationality is pre-constructed, which can represent the facial feature description under general aesthetics. For example, the lip color generally recognized by people is red, not black, and red lips are the content of the facial feature description mechanism. In other words, the facial feature description mechanism represents the general cognition and general aesthetics of the vast majority of people on facial features. Next, it is necessary to comprehensively establish neighboring descriptors and non-neighboring descriptors of each feature attribute based on the facial feature description mechanism and the style requirements for the target model face. Since the facial feature description mechanism only represents general cognition, there are some models with special styles, and it is precisely because of the special characteristics of their faces that they have a sense of high-end. Therefore, this embodiment takes this part of the model face with special styles into consideration. If the target model has a special style, the model style accounts for a larger proportion, the proportion of the facial feature description mechanism is appropriately reduced, and even part of the description is modified. Finally, the difference between neighboring descriptors and non-neighboring descriptors is strengthened through the token processing mechanism, so that the weight between neighboring descriptors is increased and the weight between non-neighboring descriptors is reduced, the description of feature attributes by non-neighboring descriptors is reduced, and the probability of description confusion is reduced.

[0132] After obtaining the reference model face and the source model face, you can follow the Figure 7 The process shown is then followed by Figure 7 The customized model face in is the output target model face. For the source model face after face analysis, an encoder is used for processing to obtain its feature vector expression. For the reference model face image, a convolutional neural network (such as ResNet) is used for processing, and the feature expression of the reference model face is also extracted. Preferably, the Arcface loss function is used for the reference model face image to better achieve inter-class separation and intra-class aggregation of the model face, which helps to convert the key information in the image into a feature vector, which can effectively characterize the characteristics of the model face image. The reference model face feature vector and the source model face feature vector are decoupled and aggregated, and the features are filled based on the regions delineated in the facial data.

[0133] Specifically, the feature vectors of the source model face and the reference model face are decoupled and aggregated. Decoupling realizes the separation of the source model face features from the reference model face features; aggregation realizes the merging of the source model face features and the reference model face features, thereby obtaining a new latent space representation. This new latent space representation contains both the features of the source model face and the features of the reference model face, and also integrates other representation information of the source model face and the reference model face. The latent space representation stores relevant features in the form of feature vectors.

[0134] The features corresponding to the feature vector in the latent space representation specifically include significant features, features merged from significant features of the same feature attribute on the source model face and the reference model face, and features merged from non-significant features of the same feature attribute on the source model face and the reference model face. For the same feature attribute, significant features can exist only on the source model face, only on the reference model face, or even on both the source model face and the reference model face. When significant features only exist on the source model face or the reference model face, there is no need to perform feature fusion, and only the significant features need to be customized to the target model face. When both the source model face and the reference model face have significant features for the same attribute feature (such as a feature of the eye), in order to fully retain the two significant features, the scheme of this embodiment fuses the significant features on the source model face and the reference model face, and the characteristics of both are retained in the fused features, and then the fused features are customized to the target model face. For non-significant features, fusion can be performed, or one can be retained, which can be set according to actual conditions.

[0135] After that, the decoder is used to decode the feature vector after feature fusion, and finally the target model face result that meets the characteristics of the source model face and the reference model face is obtained, which effectively retains the consistency information of the model's facial features and realizes the customization of the model's facial features. In addition, in order to further meet the higher quality model face customization, the face beautification module is used to upgrade the target model face effect. Specifically, super-resolution, detail optimization and enhancement, and beauty algorithms are used to optimize and improve the recognizability and expressiveness of the model face, making the contour, facial features, skin color and other features of the model face more refined and vivid.

[0136] Based on the use of the model face consistency information, the model face customization solution of this embodiment can also realize the model face customization of batch images such as group pictures, sets of pictures, image sequences, and videos.

[0137] This embodiment proposes a model face customization method based on maintaining facial consistency. Given a source model face and a reference model face, a target model face can be quickly output, and the feature consistency of the source model face and the reference model face can be effectively maintained, thereby achieving high-quality output of the model face, and having high image resolution and rich model face details, and accurately modifying and customizing facial features.

[0138] [Example 4]

[0139] This embodiment proposes a model face customization system based on maintaining facial consistency, which systematizes the model face customization method based on maintaining facial consistency in Embodiment 3 to make it more practical. The overall structure diagram of the model face customization system is shown in the attached manual. Figure 8 As shown, the system includes:

[0140] A first acquisition unit 81 is used to acquire a source model face and extract facial data and / or hair data of the source model face;

[0141] The second acquisition unit 82 directly acquires the reference model face; or uses a preset face parameterization customization module to perform parameterization description from one or more dimensions of basic features, facial features and hair features to obtain the reference model face;

[0142] A feature vector unit 83 extracts a first feature vector from the facial data and / or hair data of the source model face, and performs face recognition on the reference model face to extract a second feature vector;

[0143] A feature fusion unit 84 performs feature fusion on the first feature vector and the second feature vector to obtain a latent space representation;

[0144] The generating unit 85 is used to decode the latent space representation to obtain the target model face, and use the face beautification module to optimize and improve the recognizability and expressiveness of the target model face.

[0145] Among them, in the face parameter customization module, the various feature attributes on the reference model's face are described through descriptive words to achieve parametric description; and a token processing mechanism is pre-built to deal with the feature attribute description confusion caused by the coupling relationship between the feature attributes regarding the descriptive words.

[0146] The model face customization scheme based on maintaining facial consistency proposed in the aforementioned embodiments 3 and 4 can quickly output a fused target model face for a given source model face and a reference model face, and can effectively maintain the feature consistency of the source model face and the reference model face, retain the personalized features on the model face, achieve high-quality output of the model face, and have high image resolution and rich details of the model face, and can accurately modify and customize the appearance features. The face parameterized customization module can meet the highly personalized customization of the reference model face, thereby realizing the application of more demand scenarios.

[0147] As mentioned above, Example 1 and Example 2 disclose a personalized customized image output scheme, and Example 3 and Example 4 disclose a model face customization scheme based on maintaining facial consistency. Among them, in the personalized customized image output scheme, the rendered image can be modified by adjusting the model elements to obtain the finished image. The model face customization scheme can fuse the features of the source model face and the reference model face by feature fusion to generate the target model face. This scheme can be regarded as a specific implementation of "modifying the rendered image by adjusting the model elements" in the model face scenario.

[0148] In view of the fact that the visible parts of the model in the rendered image in the aforementioned personalized customized rendering scheme include not only the face but also the limbs (such as hair, neck, limbs, waist, etc.), the present invention also proposes another personalized customized rendering scheme based on image fusion, that is, determining the face image according to the face image uploaded by the user or according to the face description information specified by the user, and then fusing the face image with the first dressing effect image to obtain the second dressing effect image, thereby extending the feature fusion technology shown in Examples 3 and 4 to the whole body of the digital model. The personalized customized rendering scheme is described below in conjunction with Examples 5 and 6.

[0149] [Example 5]

[0150] This embodiment proposes a new personalized customized image output method, which can be applied to the personalized customized image output system shown in Example 6 below. The system can include only a client or a client and a server.

[0151] In one embodiment, the method can be applied to a 3D model processing software, which can be used to process a 3D model and output an image corresponding to the 3D model. Wherein, if the 3D model processing software supports the 3D modeling function, the processed 3D model can be generated by the software through the function; and if the 3D model processing software does not support the 3D modeling function, the software can receive a 3D model constructed and output by other 3D modeling software, process the model locally, and output its corresponding image.

[0152] In another embodiment, the method can be applied to three-dimensional clothing model processing software, which is used to output a dressing effect image based on a three-dimensional clothing model and a digital model. Among them, the three-dimensional clothing model processed by the software can be jointly modeled by a three-dimensional clothing model (obtained by modeling the clothing) and a digital model (obtained by modeling the physical model). Similar to the aforementioned three-dimensional model processing software, when the three-dimensional clothing model processing software supports the three-dimensional modeling function, the processed three-dimensional clothing model can be generated by the software through the function; and when the three-dimensional clothing model processing software does not support the three-dimensional modeling function, the software can receive a three-dimensional clothing model constructed and output by other three-dimensional modeling software, process the model locally, and output its corresponding dressing effect image. Specifically, the clothing used to construct the three-dimensional clothing model can be the target clothing described in this embodiment. Exemplarily, the three-dimensional clothing model processing software can be Style3D Studio or other software.

[0153] It should also be noted that the personalized drawing system can belong to the aforementioned three-dimensional model processing software or the aforementioned three-dimensional clothing model processing software. Taking the three-dimensional clothing model processing software as an example, when the personalized drawing system only includes a client, the client is the client of the three-dimensional clothing model processing software; and when the personalized drawing system includes a client and a server, the client and the server are the client and the server of the three-dimensional clothing model processing software.

[0154] like Fig. 9 As shown, the method includes steps S910-S930.

[0155] Step S910: Determine the face description text specified by the user, and call a generative algorithm to generate a face image based on the face description text.

[0156] In this embodiment, the user can specify the face description text according to their own needs, so that the personalized image system can call the generative algorithm to generate a face image that meets the above needs based on the text. The face image generated by this method is more realistic (that is, the CG sense is weaker), which helps to ensure that the second dressing effect image obtained by the fusion process has a strong sense of authenticity. Obviously, the face image generated by this method is the image to be fused.

[0157] Among them, the client can provide the user with a corresponding operation interface, such as displaying the operation interface after the user logs in to the account through the client. The client can be a native application (i.e., Application, APP) installed on the electronic device, or it can also be a small program, quick application or other similar forms. Of course, when using web technologies such as HTML5 or similar, the above operation interface can be accessed and displayed through a browser. The browser here can be an independent browser application or a browser module embedded in certain applications.

[0158] In one embodiment, the user can specify the above-mentioned facial description text in a variety of ways. For example, the operation interface can display multiple facial description options, each of which has a corresponding facial description text. The user can select at least one facial description option according to his or her own needs; accordingly, the client can determine the facial description text corresponding to the facial description option selected by the user (the text determined in this way is referred to as the first facial description text). Of course, in the case where the user selects multiple options at the same time, the client can determine the facial description text corresponding to each of the selected options, which will not be repeated here. Fig.12a and Fig.13aAs shown, the interface displays multiple face description options such as "gender", "age", and "region", and the user can select corresponding values ​​for each option as needed. Assuming that the user selects gender as "female", age as "20-25 years old", and region as "Asia", the system can determine "female", "20-25 years old", and "Asian" as the first face description text according to the above selection results.

[0159] And / or, the operation interface may also display a text input box, in which the user may enter a face description text that meets his / her needs; accordingly, the client may receive the face description text entered by the user (the text received in this manner is referred to as the second face description text). Fig.13a As shown, the interface displays the words "Please enter description text" and a corresponding text input box. Assume that the user enters the text "long side bangs, high nose bridge, thick lips, white skin, black eyes, baby fat" in the input box, then the system can determine the text as the second face description text.

[0160] After obtaining the above-mentioned face description text (i.e., the first face description text and / or the second face description text), the system can call a generative algorithm to generate a face image based on the face description text, such as generating face prompts based on the face description text (such as directly using the text as prompts, or using the description keywords in the text and a preset prompt template to generate face prompts), and input the face prompts into the generative algorithm for reasoning operations to generate the corresponding face image. Among them, the generative algorithm can be deployed locally in the system, such as running in an electronic device where the client is located, or running on a server where the server is located; or it can be deployed on other platforms and the system can access the algorithm through an open interface, which will not be repeated. Exemplarily, the generative algorithm can be in the form of GAN or diffusion model, etc., and the embodiments of the present invention are not limited to this.

[0161] In one embodiment, the system can obtain the facial image by parametrically describing one or more dimensions of basic features, facial features, and hair features. The description can be set according to a specific facial image or according to application experience to meet the customization requirements of the facial image. It can be seen that this method can meet the highly personalized customization of facial images, and thus realize the application of more demand scenarios. For the parametric description, generative algorithms such as generative networks, such as GAN, diffusion models, etc., can be used to generate facial images that meet the feature requirements, effectively achieving the visual effect of the uniqueness and beauty of the face in the facial image.

[0162] Among them, the system can describe each characteristic attribute of the face image through descriptive words to achieve parameterized description, wherein a token processing mechanism can be pre-constructed to handle the confusion of characteristic attribute description caused by the coupling relationship between characteristic attributes regarding descriptive words. The token processing mechanism can associate facial features with general descriptions of the features, and avoid confusion problems caused by coupling between characteristic descriptions. The token mechanism establishes the difference between adjacent descriptive words and non-adjacent descriptive words, so that adjacent descriptive words have higher weights and non-adjacent descriptive words have lower weights, which is similar to the intra-class spacing and inter-class spacing of clustering problems. For example, red eyebrows and light lips, the token processing mechanism associates red with eyebrows and light with lips, and avoids confusion problems caused by coupling between characteristic descriptions such as red lips and light eyebrows.

[0163] In one embodiment, the system can pre-construct a model feature description mechanism with general rationality; then establish neighboring descriptors and non-neighboring descriptors of each feature attribute according to the model feature description mechanism and the style requirements of the fused digital model; finally, the difference between neighboring descriptors and non-neighboring descriptors is strengthened through the token processing mechanism, so that the weight between neighboring descriptors is increased and the weight between non-neighboring descriptors is reduced, and the description of feature attributes by non-neighboring descriptors is reduced, thereby reducing the probability of description confusion. Among them, the model feature description mechanism can represent the facial feature description under general aesthetics. For example, the lip color generally recognized by people is red, not black, and red lips are the content of the facial feature description mechanism. In other words, the facial feature description mechanism represents the general cognition and general aesthetics of the vast majority of people on facial features. Next, it is necessary to comprehensively establish neighboring descriptors and non-neighboring descriptors of each feature attribute based on the facial feature description mechanism and the style requirements of the facial part of the fused digital model (hereinafter referred to as the fused model face). Since the facial feature description mechanism only represents general cognition, there are some models with special styles, and it is precisely because of the special characteristics of their faces that they have a sense of luxury. Therefore, this embodiment takes these special styles of model faces into consideration. If the model's face style is special after fusion, the proportion of the model style is greater, the proportion of the facial feature description mechanism is appropriately reduced, and even part of the description is modified. Finally, the difference between neighboring descriptors and non-neighboring descriptors is strengthened through the token processing mechanism, so that the weights between neighboring descriptors are increased and the weights between non-neighboring descriptors are reduced, the description of feature attributes by non-neighboring descriptors is reduced, and the probability of description confusion is reduced.

[0164] like Fig.13aAs shown, after the user selects the above face description option and enters the corresponding face description text, the user can click the "Generate" button in the lower right corner to trigger the system to generate a corresponding face image according to the above description text. The generated result is shown as face image 1301.

[0165] Step S920: Acquire a first dressing effect image of the target garment, where the first dressing effect image is used to present the wearing effect of the target garment on a digital model.

[0166] In addition to receiving or determining the aforementioned face image, the system also needs to obtain the first dressing effect image of the target clothing as the image to be fused. Among them, the target clothing may include clothing that needs to be worn on the corresponding part of the digital model, and the clothing may be of any type, such as a hat, scarf, top, pants, skirt, shoes, etc., and the embodiment of the present invention is not limited to this. It should also be noted that the target clothing described in the present invention may specifically include at least one piece of clothing, such as the digital model may only wear a skirt or only wear a top (i.e., only wear one piece of clothing), or may also wear a hat, a top and a skirt at the same time (i.e., wear multiple pieces of clothing at the same time), etc., which will not be repeated. The first dressing effect image is used to present (after the clothing is worn on the corresponding part of the digital model) the wearing effect of the target clothing on the digital model.

[0167] In one embodiment, similar to the aforementioned face image, the first dressing effect image can also be generated in advance by the user and uploaded to the system. It can be generated in advance by editing using any form of image editing software such as PS, Style3D Studio, etc.

[0168] In another embodiment, the system may also support a three-dimensional modeling function. In this case, the system may perform three-dimensional modeling on the target clothing and the digital model to generate a three-dimensional dressing model, and render the model to obtain the first dressing effect image. In this way, the personalized customized drawing system can use its own compatible three-dimensional modeling function to generate the first dressing effect image to achieve seamless connection with subsequent fusion processing, and can realize one-stop customized drawing of the system, which helps to simplify user operations and improve the usability of the system. Of course, in the case where the system does not support the three-dimensional modeling function, it can also receive the first dressing effect image generated by the function from other software that supports the three-dimensional modeling function, which will not be repeated here.

[0169] It should be noted that the first dressing effect image may include a clothing area (i.e., the location of the target clothing), a naked body area (i.e., the body area of ​​the digital model not covered by the target clothing, such as the face, hair, neck, waist, limbs, etc.), and may also include a background area (i.e., an irrelevant area in the image other than the clothing area and the naked body area). The naked body area may only include a face area (e.g., the digital model only has a face and a torso, but no limbs); of course, in some embodiments, the naked body area may also include a limb area (i.e., the digital model includes limbs), etc., which will not be repeated.

[0170] It should also be noted that the embodiment of the present invention does not limit the execution order of steps S910 and S920, and can be flexibly adjusted according to actual needs.

[0171] Step S930: fusing the face image and the first dressing effect image to generate a second dressing effect image of the target garment, wherein the second dressing effect image is used to present the wearing effect of the target garment on the fused digital model.

[0172] After determining the face image and the first dressing effect image in the aforementioned manner, the system may fuse the two to generate a second dressing effect image for presenting the wearing effect of the target garment on the fused digital model.

[0173] It is understandable that this solution can achieve high-quality rapid output of clothing images, improve the efficiency of output and content distribution and marketing needs, maintain high fidelity of modeled clothing, and is not difficult to operate, significantly reducing time and labor costs, while also having a wide range of applications. In addition, users can specify facial description text according to their own needs to call the generative algorithm to generate a facial image that meets the needs based on the text. Compared with facial images drawn by users themselves or captured from 3D models, the facial images generated by this method are more realistic (i.e., the CG sense is weaker), which helps to ensure that the second dressing effect image obtained by the fusion process has a strong sense of authenticity. In addition, this solution can obtain a second dressing effect picture for presenting the wearing effect of the target clothing on the fused digital model by fusing the face image and the first dressing effect image for presenting the wearing effect of the target clothing on the digital model. The image can maintain the consistency of the features of the digital model in the face image and the first dressing effect image, that is, the appearance of the fused digital model in the image can present both the facial features in the above-mentioned face image and the original human features of the digital model. The display effect of the second dressing effect picture can also meet the above-mentioned needs of the user. Obviously, compared with the related technical solutions of directly cutting, splicing / replacing faces, the human body display effect in the second dressing effect picture obtained by the fusion processing of this solution is more realistic and natural.

[0174] In one embodiment, when the face image and the first dressing effect image are fused, on the one hand, a face feature vector can be extracted for the face image; on the other hand, the first dressing effect image can be parsed to identify the clothing area and the exposed human body area not covered by the target clothing, and a dressing effect feature vector can be extracted for the first dressing effect image, the vector at least including the human body feature vector corresponding to the exposed human body area; then, the face feature vector and the dressing effect feature vector can be feature fused, and the fusion result can be decoded to obtain a second dressing effect image. It can be understood that the face feature vector extracted from the face image can characterize the significant features of the face in the face image; and the human body feature vector in the dressing effect feature vector extracted from the first dressing effect image can characterize the significant features of the human body part corresponding to the aforementioned exposed human body area. By fusing the above two feature vectors, this solution can ensure that the human body part of the fused digital model in the second dressing effect image has both of the above two significant features, thereby ensuring feature consistency.

[0175] In one embodiment, the first dressing effect image can be analyzed in a variety of ways to identify the clothing area and the naked body area. For example, the first dressing effect image can be segmented using an image segmentation algorithm to identify the clothing area and the naked body area. Among them, the image segmentation algorithm can be implemented based on threshold, edge, region, cluster analysis, wavelet transform, etc., or can also be implemented based on artificial neural network or genetic algorithm, etc., and the embodiment of the present invention does not limit this. For another example, when the first dressing effect image is obtained by rendering a three-dimensional dressing model, the image can inherit (or be associated with) the corresponding three-dimensional space related information (such as depth information, curve information, light information, etc.) from the three-dimensional dressing model; based on this, the system can also identify the clothing area and the naked body area according to the three-dimensional space related information of the first dressing effect image.

[0176] In one embodiment, in view of the fact that the dressing effect feature vector includes a human feature vector corresponding to the exposed human body area, when the face feature vector and the dressing effect feature vector are feature fused, the face feature vector and the human feature vector can be feature fused. Among them, if the dressing effect feature vector also includes a clothing feature vector corresponding to the clothing area, the clothing area (and possibly the background area) in the first dressing effect image can be masked, so that the face feature vector is only feature fused with the face feature vector in the clothing effect feature vector, but not with the clothing feature vector therein. In this fusion mode, the clothing feature vector does not participate in the fusion calculation, and the display effect of the target clothing in the first dressing effect image and the second dressing effect image is completely consistent without any change.

[0177] Alternatively, the face feature vector and the dressing effect feature vector may be feature fused, that is, the face feature vector is fused with the whole body composed of the human body feature vector and the clothing feature vector. In this fusion mode, the clothing feature vector will participate in the fusion calculation, and the display effect of the target clothing in the first dressing effect image and the second dressing effect image is basically the same, except that there may be slight changes in clothing details such as angle and light. Of course, in order to ensure that the display effect of the target clothing in the first dressing effect image and the second dressing effect image is completely consistent, it is also possible to use the aforementioned clothing feature vector to replace the corresponding vector part in the fusion result after fusing the face feature vector with the whole body composed of the human body feature vector and the clothing feature vector, and then decode the replaced fusion result to obtain the second dressing effect image; or decode the fusion result after fusion to obtain a preliminary second dressing effect image, and use the aforementioned clothing area to cover the corresponding part in the image to obtain the final second dressing effect image. In the above manner, it can also be ensured that the display effect of the target clothing in the final generated second dressing effect image is completely consistent with its display effect in the first dressing effect image.

[0178] Of course, it can be seen from the aforementioned Examples 3 and 4 that the human body feature vector can also be divided into a first face feature vector (corresponding to the face part in the first dressing effect image) and a first limb feature vector (corresponding to the limb part other than the face part in the first dressing effect image), and the face feature vector is only fused with the first face feature vector, so that only the face display effect of the digital model in the first dressing effect image is changed through the fusion processing, without changing its limb display effect. The specific process will not be repeated here.

[0179] As mentioned above, the second dressing effect image can be obtained by decoding the fusion result. In one embodiment, during the decoding process, the fusion result corresponding to the human body region can be decoded to obtain the fused naked human body region, and the fused naked human body region and the clothing region can be spliced ​​to obtain the second dressing effect image. In this way, it can also be ensured that the display effect of the target clothing in the final generated second dressing effect image is completely consistent with its display effect in the first dressing effect image.

[0180] In the aforementioned embodiments, the fusion result includes a latent space representation, and the specific form of the representation can be found in the records of the aforementioned embodiments 3 and 4, which will not be repeated here. It can be understood that the latent space (or shallow space) representation can be achieved through dimensionality reduction techniques (such as principal component analysis, autoencoders, etc.), which allow complex data structures to be mapped to simpler and more compact spaces while retaining the significant features of the data as much as possible. In this low-dimensional latent space, data is generally easier to process, analyze and understand, and it also helps to visualize and generate data. Therefore, this method helps to achieve tasks such as dimensionality reduction compression, denoising, and feature extraction. Of course, when conditions such as fusion accuracy and computing resources are met, the fusion result can also be characterized by feature space (Feature space) characterization, parameter space (Parameter space) characterization, etc., and the embodiments of the present invention are not limited to this.

[0181] like Fig.10 As shown, the personalized customized drawing system can support functions such as 3D drawing and 2D drawing. The following embodiments take the 3D drawing process as an example. After the user clicks the "3D drawing" button, it jumps to Fig.11 The dressing effect picture editing interface shown in the figure. In this interface, users can adjust the size, angle, direction, background and other drawing parameters of the 3D model (such as adjusting these parameters in the "drawing parameter editing" box on the right), and trigger the system to render and output the dressing effect pictures under different perspectives, such as Fig.11 As shown, two dressing effect images 1101 and 1102 are output. In this regard, the user clicks the "Save" button to save the above dressing effect image, or can also click the "Local Rendering" button to further render and edit it, which will not be repeated. In addition, the user can select any number of the dressing effect images as the object of fusion processing (that is, as the first dressing effect image described in the embodiment of this specification), and the following embodiment takes the user selecting the dressing effect image 1101 as an example for explanation.

[0182] After selecting the dress effect image 1101, the user can click the "AI output image" button to jump to Fig.12a The face image designation interface shown or Fig.13a The face image generation interface shown.

[0183] exist Fig.12a In the face image designation interface shown, the user can achieve personalized customization of the image through a variety of operation methods. For example, the user can select appropriate face features as needed in the basic feature selection box 1201, but do not select any image in the detailed feature selection box 1202. At this time, after clicking the "Merge Image" button, the system will automatically generate a corresponding face image according to the selected face features, and then merge the face image with the image. Fig.11 The dressing effect image 1101 shown (this image is taken as the first dressing effect image) is fused to obtain a corresponding second dressing effect image.

[0184] For another example, the user can also select any standard face image provided by the system in the detail feature selection box 1202, or upload a pre-generated local face image and select the image, but do not select face features in the basic feature selection box 1201. At this time, after clicking the "Merge Image" button, the system will use the face image selected in the detail feature selection box 1202 as the fusion object, and merge the face image with the basic feature selection box 1201. Fig.11 The dressing effect image 1101 shown is fused to obtain a corresponding second dressing effect image.

[0185] For another example, the user can select appropriate facial features as needed in the basic feature selection box 1201, and select any image in the detailed feature selection box 1202. At this time, after clicking the "Merge Image" button, the system will first generate a new facial image according to the selected facial features and the selected image (the facial image is superimposed with the selected facial features on the basis of the selected image, and the superposition process can also be regarded as a kind of "feature fusion"), and then combine the facial image with the selected image. Fig.11 The dressing effect image 1101 shown is fused to obtain a corresponding second dressing effect image.

[0186] After generating the second dressing effect image, you can jump to Figure 12b The fusion effect display interface shown in the figure shows the "original effect image" and "fusion effect image" displayed in the interface. Fig.11 The dressing effect image 1101 shown in the figure and the second dressing effect image obtained by the fusion process. At this time, the user can click the "Save" button to save the fusion effect image, and of course, the original effect image and the fusion effect image can also be saved at the same time. It can be understood that after the fusion effect image is saved, the personalized customization process described in this solution is completed.

[0187] Fig.12a The basic features shown can be regarded as the facial features corresponding to the first facial description text described in the previous embodiment. In addition, the user can also flexibly edit the second facial description text. Fig.13aAs shown in the figure, in addition to selecting "gender", "age", "region" and other facial description options, the user can also enter the second facial description text "long side bangs, high nose bridge, thick lips, white skin, black eyes, baby fat" in the text input box. At this time, the user clicks the "Generate" button, and the system can automatically generate a facial image that meets the characteristics described in the first facial description text (i.e. "female", "20-25 years old" and "Asian", etc.) and the second facial description text (i.e. "long side bangs, high nose bridge, thick lips, white skin, black eyes, baby fat"), as shown in the figure. Fig.13a As shown in the face image 1301 in the figure. Of course, the user can also choose not to select the above basic features, but only directly enter the second face description text in the text input box, so that the system can call the generative algorithm to generate the corresponding face image based on the text, which will not be repeated. If the user is not satisfied with the image, the user can adjust the above face description options or face description text and regenerate a new face image until he is satisfied. If the user is satisfied with the face image, he can further click "Merge Image" in the lower right corner of the interface, and the system can then Fig.11 The first dressing effect image 1101 and Fig.13a The facial image 1301 shown is fused to obtain a corresponding second dressing effect image.

[0188] Similar to the above embodiment, after generating the second dressing effect image, it is possible to jump to Fig.13b The fusion effect display interface shown in the figure shows the "original effect image" and "fusion effect image" displayed in the interface. Fig.11 The first dressing effect image 1101 and the second dressing effect image obtained by fusion processing are shown. At this time, the user can click the "Save" button to save the fusion effect image. It can be understood that after the fusion effect image is saved, the personalized customization process described in this solution is completed.

[0189] Of course, the above-mentioned face image and / or the first dressing effect image can also be saved at the same time, so that the user can trace the generation process of the second dressing effect image, which will not be repeated. In addition, if the user is not satisfied with the fusion effect image generated this time, he can re-upload the face image, or modify the face description text and regenerate a new face image (of course, given that the generative algorithm may generate different face images according to the same prompt word, the user can also not modify the face description text, but directly click the "Generate" button to generate a new face image), and then click the "Fuse Image" button to regenerate a new fused image until he is satisfied.

[0190] It should also be noted that Figure 12b and Fig.13bIn addition to displaying the original effect image and the fusion effect image, the corresponding face image can also be displayed so that the user can clearly know the difference in the fusion processing effect and it is also convenient for the user to adjust the fusion effect.

[0191] As mentioned above, the dressing effect image can display at least one of the face image, the first dressing effect image and the second dressing effect image, so that the user can view the above images and clearly know the detailed display effect of the image, thereby facilitating the user to perform further operations.

[0192] In one embodiment, the display effect of the target clothing in the first dressing effect image and the second dressing effect image is consistent. According to the above-mentioned aggregation method and decoding method, the two display effects may be completely consistent, or there may be only slight differences in angles, light, etc., which will not be repeated. However, it should be noted that the personalized customized output scheme described in this embodiment does not change the various clothing elements (such as patterns and designs on clothing) and their attributes (such as the length, angle, position, color, etc. of the pattern) contained in the target clothing in the first dressing effect image, but only modifies the display effect of the digital model in the first dressing effect image through fusion processing.

[0193] In one embodiment, after obtaining the second wearing effect image in the aforementioned manner, the system can also optimize and improve the recognizability and expressiveness of the fused digital model. Specifically, one or more algorithms including super-resolution algorithm, detail optimization and enhancement algorithm, and beauty algorithm can be used to optimize the fused digital model to enhance the refinement of the fused digital model. Of course, other algorithms can also be used for optimization according to actual needs, which will not be described in detail.

[0194] Exemplarily, the model beautification module may include a face beautification module that only beautifies the face of the model, so that the system can use the face beautification module to optimize and improve the recognizability and expressiveness of the face of the fused digital model (i.e., the aforementioned fused model face). Specifically, the module can use one or more algorithms including the aforementioned super-resolution algorithm, detail optimization and enhancement algorithm, and beauty algorithm to optimize the fused model face to enhance the refinement of the fused model face, so that the contour, facial features, skin color and other features of the model face are more refined and vivid, thereby ensuring that the face part in the second dressing effect picture has a better detail display effect and a better sense of reality.

[0195] [Example 6]

[0196] This embodiment proposes a personalized customized image output system, which systematizes a personalized customized image output method in Embodiment 5 to make it more practical. The overall structure diagram of the system is shown in the attached manual. Fig.14As shown, the system includes:

[0197] A face image generation module 141 is used to determine a face description text specified by a user and to call a generative algorithm to generate a face image based on the face description text;

[0198] A dressing image acquisition module 142 is used to acquire a first dressing effect image of a target garment, wherein the first dressing effect image is used to present a wearing effect of the target garment on a digital model;

[0199] The image fusion module 143 is used to fuse the face image and the first dressing effect image to generate a second dressing effect image of the target clothing, and the second dressing effect image is used to present the wearing effect of the target clothing on the fused digital model.

[0200] Optionally, the face image generation module 141 specifically includes:

[0201] The text determination unit 1411 is used to determine the first face description text corresponding to the face description option selected by the user;

[0202] And / or, the text receiving unit 1412 is used to receive a second face description text input by a user.

[0203] Optionally, the clothing image acquisition module 142 specifically includes:

[0204] The model rendering unit 1421 is used to perform three-dimensional modeling on the target clothing and the digital model to generate a three-dimensional clothing model, and render the three-dimensional clothing model to obtain the first clothing effect image.

[0205] Optionally, the image fusion module 143 specifically includes:

[0206] A face vector extraction unit 1431, configured to extract a face feature vector from the face image;

[0207] The effect vector extraction unit 1432 is used to analyze the first dressing effect image to identify the clothing area and the naked human body area not covered by the target clothing, and extract the dressing effect feature vector for the first dressing effect image, wherein the dressing effect feature vector at least includes the human body feature vector corresponding to the naked human body area;

[0208] The feature vector fusion unit 1433 is used to perform feature fusion on the face feature vector and the dressing effect feature vector, and decode the fusion result to obtain a second dressing effect image.

[0209] Optionally, the effect vector extraction unit 1432 is specifically used for:

[0210] Using an image segmentation algorithm to segment the first dressing effect image to identify the clothing area and the naked body area; or,

[0211] The clothing area and the naked body area are identified according to the stereoscopic space related information of the first dressing effect image.

[0212] Optionally, the feature vector fusion unit 1433 is specifically used for:

[0213] Perform feature fusion on the face feature vector and the body feature vector.

[0214] Optionally, the feature vector fusion unit 1433 is specifically used for:

[0215] The fusion result corresponding to the human body region is decoded to obtain a fused naked human body region, and the fused naked human body region and the clothing region are spliced ​​to obtain a second dressing effect image.

[0216] Optionally, the fusion result includes a latent space representation.

[0217] Optionally, it also includes: an image display module 144, which is used to display at least one of the face image, the first dressing effect image and the second dressing effect image.

[0218] Optionally, the display effects of the target clothing in the first dressing effect image and the second dressing effect image are consistent.

[0219] Optionally, it also includes: an effect enhancement module 145, which is used to optimize and enhance the recognizability and expressiveness of the fused digital model after obtaining the second wearing effect image.

[0220] Optionally, one or more algorithms including a super-resolution algorithm, a detail optimization and enhancement algorithm, and a beautification algorithm are used to optimize the fused digital model to enhance the refinement of the fused digital model.

[0221] Optionally, the face image generation module 141 is specifically used for:

[0222] Based on the face description text, the face image is obtained by performing parameterized description from one or more dimensions of basic features, facial features and hair features.

[0223] Optionally, each characteristic attribute of the facial image is described by descriptive words to achieve parameterized description, wherein a pre-built token processing mechanism is used to handle characteristic attribute description confusion caused by the coupling relationship between characteristic attributes with respect to descriptive words.

[0224] Optionally, a feature description module 146 is also included, which is used to:

[0225] Pre-construct a model characterization mechanism with general plausibility;

[0226] Establishing neighboring descriptors and non-neighboring descriptors of each feature attribute according to the model feature description mechanism and the style requirements of the fused digital model;

[0227] The token processing mechanism is used to strengthen the difference between neighboring descriptors and non-neighboring descriptors, thereby increasing the weights between neighboring descriptors and decreasing the weights between non-neighboring descriptors, reducing the description of feature attributes by non-neighboring descriptors, and reducing the probability of description confusion.

[0228] Optionally, the system belongs to three-dimensional model processing software, and the three-dimensional model processing software is used to output an image corresponding to the three-dimensional model.

[0229] Optionally, the system belongs to three-dimensional clothing model processing software, and the three-dimensional clothing model processing software is used to output dressing effect images based on three-dimensional clothing models and digital models.

[0230] Fig.15 is a schematic structural diagram of a device provided by an exemplary embodiment. Fig.15 At the hardware level, the device includes a processor 1502, an internal bus 1504, a network interface 1506, a memory 1508, and a non-volatile memory 1510, and may also include hardware required for other functions. One or more embodiments of this specification may be implemented based on software, such as the processor 1502 reading the corresponding computer program from the non-volatile memory 1510 into the memory 1508 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0231] Based on the same concept as the above method, this specification also provides an electronic device, including: a processor; a memory for storing processor executable instructions; wherein the processor implements the steps of the method described in any of the above embodiments by running the executable instructions.

[0232] Based on the same concept as the above method, this specification also provides a computer-readable storage medium on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0233] Based on the same concept as the above method, this specification also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

Claims

1. A personalized customized drawing method, characterized in that: The method comprises: Performing three-dimensional modeling on the clothing and the model to generate a three-dimensional model, and rendering the three-dimensional model to obtain a rendered image; Analyzing clothing elements and model elements of the rendered image; The rendered image is modified by adjusting the clothing elements, the model elements and / or adding new elements to obtain a finished image.

2. The personalized customized drawing method according to claim 1, characterized in that: The method further comprises: When generating the three-dimensional model, presetting a scene suitable for the clothing and the model to generate the three-dimensional model including the scene; When parsing the clothing elements and model elements of the rendered image, the scene elements of the rendered image are also parsed, so that the rendered image can be modified by adjusting the scene elements.

3. The personalized customized drawing method according to claim 1, characterized in that: The modifying the rendered image by adjusting the clothing elements, the model elements and / or adding new elements to obtain a finished image includes: Encoding the rendered image; Modifying the encoded rendering image by adjusting the clothing element, the model element and / or adding a new element to obtain a modified image; The modified picture is decoded to obtain the finished picture.

4. The personalized customized drawing method according to claim 1 or 3, characterized in that: The modifying of the rendered image by adding new elements includes: Adding scene elements to modify the scene of the rendered image based on the scene elements; and / or, adding character elements to modify the model based on the character elements; And / or, adding text information so that the text information is displayed on the rendered image.

5. The personalized customized drawing method according to any one of claims 1 to 3, characterized in that: The model elements include the model's skin color, expression, movement, face shape, hairstyle, figure and / or makeup; The clothing elements include the silhouette, texture, color and / or style of the clothing.

6. The personalized customized drawing method according to claim 1, characterized in that: The three-dimensional modeling of the clothing and the model to generate the three-dimensional model comprises: Performing three-dimensional modeling on the garment to construct a three-dimensional garment model; Providing the user with the option of customizing a digital model, and based on the three-dimensional clothing model and the customized digital model, performing clothing simulation in the modeling software to generate the three-dimensional model; or providing the user with a plurality of mannequins for the user to choose from, simulating the three-dimensional clothing model and the mannequin selected by the user to a stable state through a simulation engine of the modeling software to generate the three-dimensional model.

7. A personalized customized drawing system, characterized in that: The system comprises: A modeling module, used for three-dimensional modeling of clothing and models to generate a three-dimensional model; A rendering module, used for rendering the three-dimensional model to obtain a rendered image; A parsing module, used for parsing clothing elements and model elements of the rendered image; The modification module is used to modify the rendered image by adjusting the clothing elements, the model elements and / or adding new elements to obtain a finished image.

8. The personalized customized drawing system according to claim 7, characterized in that: The modeling module includes: A clothing modeling unit, used for performing three-dimensional modeling on the clothing to construct a three-dimensional clothing model; A model modeling unit is used to provide the user with the option of customizing a digital model, and based on the three-dimensional clothing model and the customized digital model, perform clothing simulation in the modeling software to generate the three-dimensional model; or to provide the user with a plurality of mannequins for the user to choose from, and simulate the three-dimensional clothing model and the mannequin selected by the user to a stable state through a simulation engine of the modeling software to generate the three-dimensional model.

9. The personalized customized drawing system according to claim 7 or 8, characterized in that: The modeling module further includes a scene modeling unit, which is used to preset a scene suitable for the clothing and the model when generating the three-dimensional model, so as to generate the three-dimensional model including the scene; The parsing module comprises: A clothing parsing unit, used for parsing clothing elements of the rendered image; A model parsing unit, used for parsing model elements of the rendered image; A scene parsing unit is used to parse the scene elements of the rendered image.

10. The personalized customized drawing system according to claim 7, characterized in that: The modification module comprises: An encoder, used for encoding the rendered image; An adjustment unit, used for modifying the encoded rendering image by adjusting the clothing elements, the model elements and / or adding new elements to obtain a modified image; A decoder is used to decode the modified image to obtain the finished image.

11. A model face customization method based on maintaining facial consistency, characterized in that: include: Acquire a source model face, and extract facial data and / or hair data of the source model face; Directly obtain the reference model face; Or use a preset face parameter customization module to perform parameter description from one or more dimensions of basic features, facial features, and hair features to obtain a reference model face; Extracting a first feature vector from the facial data and / or the hair data of the source model face, and performing face recognition on the reference model face to extract a second feature vector; The first feature vector and the second feature vector are subjected to feature fusion, and the fusion result is decoded to obtain a target model face.

12. The model face customization method according to claim 11, characterized in that: Also includes: After obtaining the target model face, a face beautification module is used to optimize and improve the recognizability and expressiveness of the target model face.

13. The model face customization method according to claim 11, characterized in that: In the face parameterized customization module, each characteristic attribute on the reference model's face is described by a description word to achieve parameterized description; A token processing mechanism is pre-built to handle feature attribute description confusion caused by the coupling relationship between feature attributes regarding descriptive words.

14. The model face customization method according to claim 13, characterized in that: Pre-build a facial feature description mechanism that is generally reasonable; Establishing neighbor descriptors and non-neighbor descriptors of each feature attribute according to the face feature description mechanism and the style requirements of the target model's face; The token processing mechanism is used to strengthen the difference between neighboring descriptors and non-neighboring descriptors, thereby increasing the weights between neighboring descriptors and decreasing the weights between non-neighboring descriptors, reducing the description of feature attributes by non-neighboring descriptors, and reducing the probability of description confusion.

15. The model face customization method according to claim 11, characterized in that: The feature fusion specifically includes: Firstly, the first feature vector and the second feature vector are decoupled to separate the identity features and attribute features of the source model face and the reference model face; The first feature vector and the second feature vector are then aggregated to merge the attribute features of the source model's face with the identity features of the reference model's face.

16. The method for customizing a model face according to claim 11, characterized in that: The basic characteristics include age, gender, skin color, expression, makeup, facial details, facial contours and model style; The facial features include eyebrows, mouth, eyes, nose, and ears; The hair characteristics include hair style, hair color, and hair length.

17. The model face customization method according to claim 13, characterized in that: Selecting salient features on the source model face and the reference model face respectively, wherein the salient features include identity features of the source model face and identity features of the reference model face; The features corresponding to the feature vector in the latent space representation specifically include the salient features, the features obtained by fusing the salient features of the source model face and the reference model face for the same feature attribute, and the features obtained by fusing the non-salient features of the source model face and the reference model face for the same feature attribute.

18. The method for customizing a model face according to claim 12, characterized in that: The face beautification module uses one or more algorithms including a super-resolution algorithm, a detail optimization and enhancement algorithm, and a beauty algorithm to optimize the target model's face to enhance the refinement of facial features.

19. The method for customizing a model face according to any one of claims 11 to 18, characterized in that: The fusion result is a latent space representation.

20. A model face customization system based on facial consistency maintenance, characterized in that: include: A first acquisition unit, used to acquire a source model face, and extract facial data and / or hair data of the source model face; The second acquisition unit directly acquires the reference model face; or adopts a preset face parameterization customization module to perform parameterization description from one or more dimensions of basic features, facial features and hair features to obtain the reference model face; A feature vector unit, extracting a first feature vector from the facial data of the source model face and / or the hair data, and performing face recognition on the reference model face to extract a second feature vector; A feature fusion unit, performing feature fusion on the first feature vector and the second feature vector; The generation unit is used to decode the fusion result of the feature fusion to obtain the target model face.

21. An electronic device, characterized in that: include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method according to any one of claims 1-6 and 11-19 by executing the executable instructions.

22. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the instructions are executed by a processor, the steps of the method as claimed in any one of claims 1-6 and 11-19 are implemented.

23. A computer program product, characterized in that The method comprises a computer program / instruction which, when executed by a processor, implements the steps of the method as claimed in any one of claims 1 to 6 and 11 to 19.

Images