Virtual fitting model generation method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence, and particularly relates to a virtual fitting model generation method and device, equipment and a storage medium. According to the virtual fitting model generation method provided by the invention, target appearance information is acquired, and the target appearance information comprises a two-dimensional target image and target appearance description information; wherein the target is a human body or clothes; the two-dimensional target image and the target appearance description information are input into a content generation model, multiple two-dimensional target images output by the content generation model are obtained, and the multiple two-dimensional target images are images of the target at different angles; obtaining a three-dimensional target model according to the plurality of two-dimensional target images; according to the method, the human body model and the clothes model meeting the virtual fitting application requirements can be generated only through a small number of shot images, the virtual fitting function can be achieved only through low shooting cost and short shooting time, and convenience is created for users and virtual fitting function providers.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a method, device, equipment and storage medium for generating a virtual fitting model. Background Art

[0002] With the development of computer graphics and the continuous growth of user demand, virtual fitting functions have been widely used in scenarios such as online shopping, entertainment games, and home life.

[0003] Existing virtual fitting functions usually require generating a three-dimensional human body model and a three-dimensional clothing model based on several photos of the user's body at different angles and several photos of clothing at different angles, and then wearing the three-dimensional clothing model on the three-dimensional clothing model to observe the effect of the clothing worn on the user based on the final fitting model.

[0004] However, such a virtual fitting model generation method requires a large number of human body and clothing images and has high requirements for shooting conditions, which in turn leads to high shooting costs, making the threshold for applying the virtual fitting function in various scenarios high. Summary of the invention

[0005] In response to the above problems, the present application provides a method, device, equipment and storage medium for generating a virtual fitting model.

[0006] In a first aspect, the present application provides a method for generating a virtual fitting model, the method comprising:

[0007] Acquire target shape information, wherein the target shape information includes a two-dimensional target image and target shape description information; wherein the target is a human body or clothing;

[0008] Inputting the two-dimensional target image and the target appearance description information into a content generation model, and obtaining a plurality of two-dimensional target images output by the content generation model, wherein the plurality of two-dimensional target images are images of the target at different angles;

[0009] A three-dimensional target model is obtained according to the multiple two-dimensional target images.

[0010] Optionally, the target is a human body, and the step of inputting the two-dimensional target image and the target appearance description information into a content generation model to obtain a plurality of two-dimensional target images output by the content generation model includes:

[0011] Acquire a head and face image and a body image from the two-dimensional target image;

[0012] Inputting the head and face image and the corresponding head and face appearance description information into a first content generation model, and obtaining a plurality of two-dimensional head and face images output by the first content generation model;

[0013] The body image and the corresponding body shape description information are input into a second content generation model, and a plurality of body images output by the second content generation model are obtained.

[0014] Optionally, obtaining a three-dimensional target model according to the multiple two-dimensional target images includes:

[0015] Inputting the plurality of two-dimensional head and face images into a face reconstruction model to obtain a head and face model of the human body output by the face reconstruction model;

[0016] Acquire body feature parameters according to the plurality of two-dimensional body images, and acquire a body model of the human body according to the body feature parameters;

[0017] A three-dimensional object model is obtained according to the head and face model and the body model.

[0018] Optionally, the target is clothing, and obtaining a three-dimensional target model according to the multiple two-dimensional target images includes:

[0019] Acquire a point cloud image and a texture image of the clothing according to the plurality of two-dimensional clothing images;

[0020] Reconstructing a mesh of the clothing according to the point cloud image to obtain a mesh structure of the clothing;

[0021] Adding the texture map to the grid structure to obtain an initial three-dimensional clothing model, wherein the initial three-dimensional clothing model includes a clothing part and a mannequin part;

[0022] The mannequin part in the initial three-dimensional clothing model is removed to obtain a three-dimensional clothing model including the clothing part.

[0023] Optionally, removing the mannequin part in the initial three-dimensional clothing model to obtain a three-dimensional clothing model including the clothing part includes:

[0024] Determining the coordinates of the grid vertices to be split on the initial three-dimensional clothing model according to the preset split grid vertex coordinates;

[0025] Connecting the vertex coordinates along the mesh edge in a preset order to obtain a target model seam;

[0026] Acquire a mannequin model corresponding to the initial three-dimensional clothing model, and perform UV texture unfolding on the initial three-dimensional clothing model and the mannequin model along the seam of the target model, respectively, to obtain an unfolded view of the three-dimensional clothing model and an unfolded view of the mannequin model;

[0027] According to the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model, the mannequin part in the initial three-dimensional clothing model is removed to obtain a three-dimensional clothing model including the clothing part.

[0028] Optionally, removing the mannequin part in the initial three-dimensional clothing model according to the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model to obtain the three-dimensional clothing model including the clothing part comprises:

[0029] Compare the unfolded image of the three-dimensional clothing model and the unfolded image of the mannequin model pixel by pixel, and obtain coordinates of identical pixels;

[0030] According to the correspondence between the pixel coordinates and the mesh vertex coordinates, determining the mesh to be removed corresponding to the coordinates of the same pixel;

[0031] The meshes to be removed in the initial three-dimensional clothing model are deleted to obtain a three-dimensional clothing model including the clothing part.

[0032] Optionally, the method further includes:

[0033] Obtaining the circumference information of the three-dimensional human body model, and obtaining the target clothing size corresponding to the circumference information according to the preset corresponding relationship between different circumference information and clothing sizes;

[0034] Acquire a target three-dimensional clothing model corresponding to the target clothing size;

[0035] The target three-dimensional clothing model is worn on the three-dimensional human body model to obtain a virtual fitting model.

[0036] In a second aspect, the present application provides a virtual fitting model generation device, the device comprising:

[0037] An acquisition module, used for acquiring target shape information, wherein the target shape information includes a two-dimensional target image and target shape description information; wherein the target is a human body or clothing;

[0038] A communication module, used for inputting the two-dimensional target image and the target appearance description information into a content generation model, and obtaining a plurality of two-dimensional target images output by the content generation model, wherein the plurality of two-dimensional target images are images of the target at different angles;

[0039] A generating module is used to obtain a three-dimensional target model according to the multiple two-dimensional target images.

[0040] In a third aspect, the present application provides a virtual fitting model generation device, comprising: a memory and at least one processor;

[0041] The memory is used to store computer programs;

[0042] The at least one processor is configured to run the computer program stored in the memory to implement the method according to the first aspect.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon;

[0044] When the computer program is executed by a processor, the method according to the first aspect is implemented.

[0045] The virtual fitting model generation method provided in the present application obtains target shape information, wherein the target shape information includes a two-dimensional target image and a target shape description information; wherein the target is a human body or clothing; the two-dimensional target image and the target shape description information are input into a content generation model to obtain a plurality of two-dimensional target images output by the content generation model, wherein the plurality of two-dimensional target images are images of the target at different angles; a three-dimensional target model is obtained based on the plurality of two-dimensional target images; only a small number of captured images are required to generate a human body model and a clothing model that meet the requirements of the virtual fitting application, and the virtual fitting function can be realized with only a low shooting cost and a short shooting time, thereby creating convenience for users and providers of virtual fitting functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] Figure 1 Schematic diagram of the process of generating a virtual fitting model provided in this application Figure 1 ;

[0048] Figure 2 Schematic diagram of the process of generating a virtual fitting model provided in this application Figure 2 ;

[0049] Figure 3 A schematic diagram of the structure of a virtual fitting model generation device provided in this application;

[0050] Figure 4 A schematic diagram of the structure of a virtual fitting model generation device provided in this application.

[0051] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein, for example.

[0054] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] With the development of computer graphics and the continuous growth of user demand, virtual fitting functions have been widely used in scenarios such as online shopping, entertainment games, and home life.

[0056] Existing virtual fitting functions usually require generating a three-dimensional human body model and a three-dimensional clothing model based on several photos of the user's body at different angles and several photos of clothing at different angles, and then wearing the three-dimensional clothing model on the three-dimensional clothing model to observe the effect of the clothing worn on the user based on the final fitting model.

[0057] However, such a virtual fitting model generation method requires a large number of human body and clothing images and has high requirements for shooting conditions, which leads to high manpower and material costs for taking photos. This makes the threshold for applying the virtual fitting function in various scenarios high. Moreover, when taking human body images, users are required to make a lot of cooperation according to the shooting requirements, resulting in a poor user experience.

[0058] In view of the above problems, the present invention proposes the following technical concept: take a small number of clothing photos and human body photos, input the clothing photos and human body photos into the content generation model respectively, and input the appearance description information of the clothing and the human body respectively along with the photos, obtain multiple two-dimensional clothing images and two-dimensional human body images generated by the content generation model according to the small number of photos and text description information, generate a three-dimensional human body model according to the multiple two-dimensional human body images, and generate a three-dimensional clothing model according to the multiple two-dimensional clothing images; and when virtually trying on clothes, wear the clothing model on the human body model to simulate the visual effect when trying on clothes; by using the content generation model, only a small number of captured images are needed to generate relatively restored human body models and clothing models, thereby realizing the virtual trying on function under the premise of lower shooting cost and shorter shooting time, and improving user experience.

[0059] Artificial Intelligence Generated Content (AIGC) is a method of generating content based on artificial intelligence technology, which has a wide range of applications in the field of image generation. Image generation refers to the generation of new images through computer algorithms and models. These images may be completely fictional, artistic creations, or modified and enhanced based on existing images.

[0060] Figure 1 A method for generating a virtual fitting model provided in an embodiment of the present application; Figure 1 As shown, the virtual fitting model generation method provided by the present application includes:

[0061] S101: Obtain target shape information.

[0062] The target shape information includes a two-dimensional target image and target shape description information; the target is a human body or clothing; the target shape description information can be text information or voice audio information.

[0063] Specifically, the model generation device obtains target appearance information; for a human target, the target appearance information includes one or more two-dimensional full-body images and description information of the human appearance characteristics, such as hairstyle, skin color, makeup, etc.; for a clothing target, the target appearance information includes one or more two-dimensional clothing images and description information of the clothing appearance characteristics, such as material, thickness, etc.; the target appearance description information can be input by a user through a human-computer interaction device of the model generation device, and the two-dimensional target image can be obtained from an image acquisition device that establishes a communication connection with the model generation device; the target appearance information can also be obtained from a user terminal that establishes a communication connection with the model generation device.

[0064] S102: Input the two-dimensional target image and the target appearance description information into a content generation model, and obtain a plurality of two-dimensional target images output by the content generation model.

[0065] Among them, the multiple two-dimensional target images are images of the target at different angles; the content generation model is an artificial intelligence content generation model that has been pre-trained based on the needs of three-dimensional model generation, and can obtain condition information and target features from the two-dimensional target images and the target appearance description information, and then generate image samples of a specific category according to the conditions given by the two-dimensional target images and the target appearance description information, so that the images generated by the content generation model can meet the image standards required by the subsequent three-dimensional model generation process.

[0066] Specifically, the model generation device inputs the acquired two-dimensional target image and the target appearance description information into the content generation model, and obtains multiple two-dimensional target images output by the content generation model; when the target appearance description information is voice and audio information, the model generation device can parse the voice and audio information through voice recognition technology, convert the voice and audio information into text information, and input the converted text information into the content generation model.

[0067] S103: Obtain a three-dimensional target model according to the multiple two-dimensional target images.

[0068] Specifically, the model generating device obtains the feature information required for model generation from the multiple two-dimensional target images according to the design requirements of the three-dimensional target model and according to a preset information extraction algorithm, and generates the three-dimensional target model according to the feature information and according to a preset model generating algorithm; for a three-dimensional human body model, the generating algorithm may be, for example, a human body model generating algorithm based on parametric modeling, or a human body modeling method based on partial geometric features; for a three-dimensional clothing model, the generating algorithm may be, for example, a three-dimensional model generating method based on motion structure recovery. This embodiment does not limit the specific generating algorithm of the three-dimensional target model.

[0069] The virtual fitting model generation method provided in the present embodiment, by inputting a small number of captured two-dimensional target images and target appearance description information into a content generation model, obtains multiple two-dimensional target images at different angles output by the content generation model, and generates a three-dimensional human body model and a three-dimensional clothing model according to the multiple two-dimensional target images at different angles for subsequent virtual fitting. Such a virtual fitting model generation method only requires a small number of captured images to generate a human body model and a clothing model that meet the requirements of the virtual fitting application, and only requires a relatively low shooting cost and a relatively short shooting time to realize the virtual fitting function, which creates convenience for users and providers of virtual fitting functions.

[0070] Figure 2 Schematic diagram of the process of generating a virtual fitting model provided in the embodiment of the present application Figure 2 ;like Figure 2 As shown, the virtual fitting model generation method provided in the embodiment of the present application includes:

[0071] S201: Obtain target shape information.

[0072] The target shape information includes a two-dimensional target image and target shape description information; the target is a human body or clothing.

[0073] The implementation method of step S201 is similar to the implementation method of step S101, and will not be described in detail in this embodiment.

[0074] S202: When the target is a human body, a head and face image and a body image are acquired from the two-dimensional target image.

[0075] Specifically, the model generation device obtains the head and face image and the body image from the two-dimensional human body image according to a preset image segmentation algorithm. The image segmentation algorithm can, for example, be to crop the head and face image and the body image from the two-dimensional human body image according to a preset position and size; or it can be to identify the head and face part and the body part in the two-dimensional human body image through image recognition technology, and crop the parts to obtain the head and face image and the body image.

[0076] S203: Input the head and face image and the corresponding head and face appearance description information into a first content generation model, and obtain a plurality of two-dimensional head and face images output by the first content generation model.

[0077] Specifically, the model generation device inputs the obtained head and face image and the corresponding head and face appearance description information into the first content generation model, and sends an instruction to the first content generation model to instruct the first content generation model to generate multiple views of the head and face based on the input content, and obtain multiple two-dimensional head and face images output by the first content; when the head and face appearance description information is voice and audio information, the model generation device can parse the voice and audio information through voice recognition technology, convert the voice and audio information into text information, and input the converted text information into the content generation model.

[0078] For example, the model generation device sends a frontal image of a woman's head and face to the dialog box of the first content generation model, and continues to send text information into the dialog box with the content "The skin color of the woman's head and face in the image is yellowish to white, her pupil color is dark brown, her hairstyle is brown bun, she has no makeup, and there is a black mole under her eye. Please generate multiple views of the woman's head and face from multiple different angles." Subsequently, the multiple views of the head and face generated by the first content generation model according to the input content are obtained from the dialog box.

[0079] S204: Input the body image and the corresponding body shape description information into a second content generation model, and obtain a plurality of body images output by the second content generation model.

[0080] Specifically, the model generation device inputs the obtained body image and the corresponding body shape description information into the second content generation model, and sends instructions to the second content generation model to instruct the second content generation model to generate multiple views of the body based on the input content, and obtain multiple two-dimensional body images output by the first content; when the body shape description information is voice and audio information, the model generation device can parse the voice and audio information through voice recognition technology, convert the voice and audio information into text information, and input the converted text information into the content generation model.

[0081] Continuing with the above example, the model generation device sends the front image of the woman's body to the dialog box of the second content generation model, and continues to send text information into the dialog box with the content "The woman in the picture is of medium to thin body shape, with broad and flat shoulders, and well-developed calf muscles. Please generate multiple views of the woman's body from different angles based on this." Subsequently, the multiple views of the woman's body generated by the second content generation model according to the input content are obtained from the dialog box.

[0082] There is no clear timing relationship between step S203 and step S204. Step S203 can be executed first, and then step S204; step S204 can be executed first, and then step S203; step S203 and step S204 can also be executed at the same time; this embodiment does not limit the timing relationship between step S203 and step S204.

[0083] S205: Input the plurality of two-dimensional head and face images into a face reconstruction model, and obtain the head and face model of the human body output by the face reconstruction model.

[0084] Specifically, the model generation device inputs the obtained multiple two-dimensional head and face images into the face reconstruction model, so that the face reconstruction model generates a head and face model based on the multiple two-dimensional head and face images, and obtains the head and face model of the human body output by the face reconstruction model. The face reconstruction model can be, for example, a pre-trained DECA model.

[0085] S206: Acquire body feature parameters according to the plurality of two-dimensional body images, and acquire a body model of the human body according to the body feature parameters.

[0086] Specifically, the model generating device identifies multiple feature points including skin vertices and joint points in the multiple two-dimensional body images according to a preset feature point identification algorithm, and obtains the positions of the feature points and the vector angle relationship formed by the lines connecting the feature points, and obtains the body feature parameters of the human body corresponding to the two-dimensional body image according to the preset correspondence between the positions and the vector angle relationship and the body feature parameters; the body feature parameters may include body shape parameters and posture parameters, and the body shape parameters may be, for example, multiple parameters for indicating the height, weight, head-to-body ratio, etc. of the human body, and the posture parameters may be, for example, multiple parameters representing the overall movement posture of the human body and the relative angles of the joints; the model generating device inputs the body feature parameters into a human body modeling model, and obtains a body model of the human body generated by the human body modeling model, and the human body modeling model may be, for example, an SMPL parameterized human body model.

[0087] There is no clear timing relationship between step S205 and step S206. Step S205 can be executed first and then step S206; step S205 can be executed first and then step S206; step S205 and step S206 can also be executed at the same time; this embodiment does not limit the timing relationship between step S205 and step S206.

[0088] S207: Obtain a three-dimensional target model according to the head and face model and the body model.

[0089] Specifically, the model generation device performs edge cutting on the body model and the head and face model according to preset cutting edge mesh vertices to obtain the cut head and face model and body model; and sews the cut head and face model and the body model according to the connection relationship of the preset edge mesh vertices to obtain a complete three-dimensional human body model.

[0090] S208: When the target is clothing, the clothing image and corresponding clothing appearance description information are input into a third content generation model, and a plurality of two-dimensional clothing images output by the third content generation model are obtained.

[0091] Specifically, the model generation device inputs the obtained clothing image and the corresponding clothing appearance description information into the third content generation model, and sends instructions to the third content generation model to instruct the third content generation model to generate multiple views of the clothing according to the input content, and obtain multiple two-dimensional clothing images output by the third content; when the clothing appearance description information is voice audio information, the model generation device can parse the voice audio information through voice recognition technology, convert the voice audio information into text information, and input the converted text information into the content generation model.

[0092] For example, the model generation device sends the front and back images of a long skirt worn on a mannequin to the dialog box of the third content generation model, and continues to send text information into the dialog box with the content "The white long skirt in the picture is worn on a gray mannequin. It is made of silk and has a bow-shaped tie on the chest. The material of the tie is translucent organza. Please generate multiple views of the long skirt worn on the mannequin from multiple different angles." Subsequently, the multiple views of the long skirt generated by the third content generation model according to the input content are obtained from the dialog box.

[0093] It can be understood that in the head and face shape description information, body shape description information and / or clothing shape description information involved in the aforementioned steps, the shape features of the described objects can be customized according to the user's intention, and do not have to be completely consistent with the corresponding shape features of the input image, so that the user can instruct the content generation model to generate an image that meets his or her own preferences or needs, and then obtain a three-dimensional human body model and a three-dimensional clothing model that meet his or her own preferences or needs in subsequent steps, thereby realizing personalized fitting content customization and enhancing the fun and practicality of the virtual fitting function.

[0094] S209: Acquire a point cloud image and a texture image of the clothing according to the plurality of two-dimensional clothing images.

[0095] Specifically, the model generation device parses the multiple two-dimensional clothing images, extracts feature points in the images, compares and matches the feature points in different images obtained by analysis, obtains the correspondence between the feature points in different images, constructs a sparse point cloud map of the clothing through the SFM algorithm, obtains the camera parameters corresponding to the image, obtains the texture map of the clothing from the two-dimensional clothing image, and constructs a dense patch point cloud map according to the CMVS+PMVS algorithm.

[0096] S210: Reconstructing a mesh of the clothing according to the point cloud image to obtain a mesh structure of the clothing.

[0097] Specifically, the model generation device reconstructs the grid of the dense point cloud image, first generates a coarse grid structure, and then refines the coarse grid structure to obtain a fine grid structure of the clothing.

[0098] S211: Add the texture map to the grid structure to obtain an initial three-dimensional clothing model.

[0099] The initial three-dimensional clothing model includes a clothing part and a mannequin part.

[0100] Specifically, the model generating device adds the texture map to the corresponding grid according to the mapping relationship between different texture maps and grids, and finally obtains the initial three-dimensional clothing model.

[0101] S212: removing the mannequin part in the initial three-dimensional clothing model to obtain a three-dimensional clothing model including the clothing part.

[0102] It can be understood that since the clothing image content used for the three-dimensional reconstruction of the model is the clothing worn on the mannequin, after the three-dimensional reconstruction of the model, the initial three-dimensional clothing graphic obtained includes the clothing part and the mannequin part; but in the process of virtual fitting, the clothing part needs to be worn on the user's body model, that is, the fitting process also needs to use a three-dimensional clothing model that does not include the mannequin part. Therefore, the mannequin part in the three-dimensional clothing model can be removed to obtain a three-dimensional clothing model including the clothing part.

[0103] Optionally, the specific method steps for removing the mannequin part in the initial three-dimensional clothing model may be, for example:

[0104] Step 1: According to preset split mesh vertex coordinates, determine the mesh vertex coordinates to be split on the initial three-dimensional clothing model.

[0105] Specifically, the model generation device obtains the mesh vertices on the initial three-dimensional clothing model that are closest to each of the split mesh vertices according to the preset split mesh vertex coordinates, and obtains the coordinates of the mesh vertices that are closest as the coordinates of the mesh vertices to be split.

[0106] Step 2: Connect the vertex coordinates along the mesh edge in a preset order to obtain the target model seam.

[0107] Specifically, the model generation device connects the vertex coordinates along the edge of the mesh to be split according to a preset connection sequence, and uses the connection as the target model seam.

[0108] Step 3: Obtain a mannequin model corresponding to the initial three-dimensional clothing model, and perform UV texture unfolding on the initial three-dimensional clothing model and the mannequin model along the seams of the target model to obtain unfolded views of the three-dimensional clothing model and the mannequin model.

[0109] Specifically, the model generating device obtains a mannequin model having the same attribute information as the mannequin according to the attribute information of the mannequin contained in the initial three-dimensional model, and respectively obtains UV maps of the initial three-dimensional clothing model and the mannequin model on multiple surfaces divided by the seams of the target model as the unfolded views of the three-dimensional clothing model and the mannequin model.

[0110] Step 4: According to the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model, the mannequin part in the initial three-dimensional clothing model is removed to obtain a three-dimensional clothing model including the clothing part.

[0111] Optionally, based on the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model, the method for removing the mannequin part in the initial three-dimensional clothing model may be, for example: comparing the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model pixel by pixel, and obtaining the coordinates of the same pixels; determining the grids to be removed corresponding to the coordinates of the same pixels based on the correspondence between the pixel coordinates and the grid vertex coordinates; and deleting the grids to be removed in the initial three-dimensional clothing model to obtain the three-dimensional clothing model including the clothing part.

[0112] Specifically, the model generating device obtains the brightness value of each channel of each pixel in the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model, and compares the brightness values ​​of each channel of pixels at the same coordinate position on the two unfolded views pixel by pixel, and obtains the coordinates of pixels with the same brightness values ​​of each channel; obtains the grid vertex coordinates corresponding to the coordinates of the same pixel according to the correspondence between the two-dimensional pixel coordinates and the three-dimensional grid vertex coordinates, and uses the grid corresponding to the grid vertex coordinates as the grid to be removed; deletes the grid to be removed in the initial three-dimensional clothing model, and the remaining model part is the three-dimensional clothing model including the clothing part.

[0113] S213: Obtaining the circumference information of the three-dimensional human body model, and obtaining a target clothing size corresponding to the circumference information according to a preset correspondence between different circumference information and clothing sizes.

[0114] Understandably, in common fitting scenarios, people usually choose clothing of appropriate size to try on according to their own body shape. Therefore, in the process of virtual fitting, it is also possible to select clothing models of appropriate size for virtual fitting according to the circumference information of the three-dimensional model.

[0115] Specifically, the model generating device obtains the circumference information of the three-dimensional human body model, where the circumference information may include indicators such as height, weight, chest circumference, waist circumference and / or hip circumference, and obtains the target clothing size corresponding to the circumference information based on the correspondence between different numerical ranges of each indicator in the pre-set circumference information and clothing size.

[0116] S214: Obtain a target three-dimensional clothing model corresponding to the target clothing size.

[0117] Specifically, the storage device of the model generating device stores three-dimensional clothing models corresponding to each size of each piece of clothing. The model generating device obtains, based on the identification of the target clothing selected by the user to try on and the target clothing size, the three-dimensional clothing model corresponding to the target clothing size from among the three-dimensional clothing models of several sizes indicated by the identification, as the target three-dimensional clothing model.

[0118] S215: Wearing the target three-dimensional clothing model on the three-dimensional human body model to obtain a virtual fitting model.

[0119] Specifically, the model generating device may be based on a preset algorithm, for example, to match the target three-dimensional clothing model with the three-dimensional model in an adaptive manner, that is, based on the pre-constructed correspondence between the grid planes of different parts of the three-dimensional clothing model and the different parts of the three-dimensional human body model, the grid planes of different parts of the three-dimensional clothing model are selectively stretched and contracted and fit to the corresponding positions of the three-dimensional human body model to obtain a virtual fitting model.

[0120] The virtual fitting model generation method provided in the embodiment of the present application generates multiple two-dimensional target images through a content generation model based on a small number of captured target images and input target appearance description information, and generates a virtual fitting model based on the multiple two-dimensional target images, which not only saves the cost of shooting a large number of target images, but also allows users to describe the target appearance according to their own preferences or needs to obtain a personalized virtual fitting model, thereby increasing the practicality and interest of the virtual fitting function; by automatically performing UV texture expansion on a mannequin model and a three-dimensional clothing model including the mannequin, and removing the mannequin part of the three-dimensional clothing model according to the expansion diagram, the workload required for model optimization is effectively reduced, making the generation process of the three-dimensional clothing model more convenient and quick; by matching the three-dimensional clothing model of the corresponding size to the three-dimensional human body model according to the circumference information of the three-dimensional human body model, the visual effect of the generated virtual fitting model is more in line with the actual fitting scene, thereby improving the user experience of the virtual fitting function.

[0121] Figure 3 A schematic diagram of the structure of a virtual fitting model generating device provided in an embodiment of the present application; Figure 3As shown, the present application provides a virtual fitting model generation device, the virtual fitting model generation device 300 comprising:

[0122] The acquisition module 301 is used to acquire target shape information, wherein the target shape information includes a two-dimensional target image and target shape description information; wherein the target is a human body or clothing;

[0123] The communication module 302 is used to input the two-dimensional target image and the target appearance description information into the content generation model, and obtain a plurality of two-dimensional target images output by the content generation model, wherein the plurality of two-dimensional target images are images of the target at different angles;

[0124] The generating module 303 is used to obtain a three-dimensional target model according to the multiple two-dimensional target images.

[0125] Optionally, the acquisition module 301 is specifically used to acquire a head and face image and a body image from the two-dimensional target image;

[0126] The communication module 302 is specifically used to input the head and face image and the corresponding head and face appearance description information into the first content generation model to obtain multiple two-dimensional head and face images output by the first content generation model; and input the body image and the corresponding body appearance description information into the second content generation model to obtain multiple body images output by the second content generation model.

[0127] Optionally, the generation module 303 is specifically used to input the multiple two-dimensional head and face images into a face reconstruction model to obtain the head and face model of the human body output by the face reconstruction model; obtain body feature parameters based on the multiple two-dimensional body images, and obtain the body model of the human body based on the body feature parameters; and obtain a three-dimensional target model based on the head and face model and the body model.

[0128] Optionally, the generation module 303 is specifically used to obtain a point cloud map and a texture map of the clothing based on multiple two-dimensional clothing images; reconstruct the mesh of the clothing based on the point cloud map to obtain a mesh structure of the clothing; add the texture map to the mesh structure to obtain an initial three-dimensional clothing model, wherein the initial three-dimensional clothing model includes a clothing part and a mannequin part; and remove the mannequin part of the initial three-dimensional clothing model to obtain a three-dimensional clothing model including the clothing part.

[0129] Optionally, the generation module 303 is specifically used to determine the coordinates of the mesh vertices to be split on the initial three-dimensional clothing model according to preset split mesh vertex coordinates; connect the vertex coordinates along the mesh edge in a preset order to obtain the target model seam; obtain the mannequin model corresponding to the initial three-dimensional clothing model, and perform UV texture unfolding on the initial three-dimensional clothing model and the mannequin model along the target model seam to obtain the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model; remove the mannequin part in the initial three-dimensional clothing model according to the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model to obtain the three-dimensional clothing model including the clothing part.

[0130] Optionally, the generation module 303 is specifically used to compare the unfolded image of the three-dimensional clothing model and the unfolded image of the mannequin model pixel by pixel, and obtain the coordinates of the same pixels; determine the grids to be removed corresponding to the coordinates of the same pixels according to the correspondence between the pixel coordinates and the grid vertex coordinates; delete the grids to be removed in the initial three-dimensional clothing model to obtain the three-dimensional clothing model including the clothing part.

[0131] Optionally, the acquisition module 301 is further used to acquire the circumference information of the three-dimensional human body model, and acquire the target clothing size corresponding to the circumference information according to the preset corresponding relationship between different circumference information and clothing sizes; and acquire the target three-dimensional clothing model corresponding to the target clothing size;

[0132] The generating module 303 is further used to put the target three-dimensional clothing model on the three-dimensional human body model to obtain a virtual fitting model.

[0133] Figure 4 A schematic diagram of the structure of a virtual fitting model generation device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the virtual fitting model generation device 400 includes: at least one processor 401 and a memory 402; in addition, the virtual fitting model generation device 400 may also have a communication interface 404 for receiving and sending instructions.

[0134] Wherein, the processor 401, the memory 402 and the communication interface 404 are connected via a bus 403;

[0135] The computer program is stored in the memory 402 and is configured to be executed by the processor 401 to implement the present application. Figure 1-Figure 2 A method for generating a virtual fitting model provided by any corresponding embodiment;

[0136] Figure 4The virtual fitting model generation device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0137] In addition, this embodiment further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the virtual fitting model generation method of the above embodiment.

[0138] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0141] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform some steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0142] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0143] So far, the technical solution of the present application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a virtual fitting model, characterized in that: The method comprises: Acquire target shape information, wherein the target shape information includes a two-dimensional target image and target shape description information; wherein the target is a human body or clothing; Inputting the two-dimensional target image and the target appearance description information into a content generation model, and obtaining a plurality of two-dimensional target images output by the content generation model, wherein the plurality of two-dimensional target images are images of the target at different angles; A three-dimensional target model is obtained according to the multiple two-dimensional target images.

2. The method according to claim 1, characterized in that The target is a human body, and the two-dimensional target image and the target appearance description information are input into a content generation model to obtain a plurality of two-dimensional target images output by the content generation model, including: Acquire a head and face image and a body image from the two-dimensional target image; Inputting the head and face image and the corresponding head and face appearance description information into a first content generation model, and obtaining a plurality of two-dimensional head and face images output by the first content generation model; The body image and the corresponding body shape description information are input into a second content generation model, and a plurality of body images output by the second content generation model are obtained.

3. The method according to claim 2, characterized in that The step of obtaining a three-dimensional target model according to the plurality of two-dimensional target images comprises: Inputting the plurality of two-dimensional head and face images into a face reconstruction model, and obtaining a head and face model of the human body output by the face reconstruction model; Acquire body feature parameters according to the plurality of two-dimensional body images, and acquire a body model of the human body according to the body feature parameters; A three-dimensional object model is obtained according to the head and face model and the body model.

4. The method according to claim 1, characterized in that The target is clothing, and obtaining a three-dimensional target model according to the multiple two-dimensional target images includes: Acquire a point cloud image and a texture image of the clothing according to the plurality of two-dimensional clothing images; Reconstructing a mesh of the clothing according to the point cloud image to obtain a mesh structure of the clothing; Adding the texture map to the grid structure to obtain an initial three-dimensional clothing model, wherein the initial three-dimensional clothing model includes a clothing part and a mannequin part; The mannequin part in the initial three-dimensional clothing model is removed to obtain a three-dimensional clothing model including the clothing part.

5. The method according to claim 4, characterized in that The removing the mannequin part from the initial three-dimensional clothing model to obtain a three-dimensional clothing model including the clothing part comprises: Determining the coordinates of the mesh vertices to be split on the initial three-dimensional clothing model according to the preset split mesh vertex coordinates; Connecting the vertex coordinates along the mesh edge in a preset order to obtain a target model seam; Acquire a mannequin model corresponding to the initial three-dimensional clothing model, and perform UV texture unfolding on the initial three-dimensional clothing model and the mannequin model along the seam of the target model, respectively, to obtain an unfolded view of the three-dimensional clothing model and an unfolded view of the mannequin model; According to the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model, the mannequin part in the initial three-dimensional clothing model is removed to obtain a three-dimensional clothing model including the clothing part.

6. The method according to claim 5, characterized in that The step of removing the mannequin part in the initial three-dimensional clothing model according to the unfolded view of the three-dimensional clothing model and the unfolded view of the mannequin model to obtain the three-dimensional clothing model including the clothing part comprises: Compare the unfolded image of the three-dimensional clothing model and the unfolded image of the mannequin model pixel by pixel, and obtain coordinates of identical pixels; According to the correspondence between the pixel coordinates and the mesh vertex coordinates, determining the mesh to be removed corresponding to the coordinates of the same pixel; The meshes to be removed in the initial three-dimensional clothing model are deleted to obtain a three-dimensional clothing model including the clothing part.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining the circumference information of the three-dimensional human body model, and obtaining the target clothing size corresponding to the circumference information according to the preset corresponding relationship between different circumference information and clothing sizes; Acquire a target three-dimensional clothing model corresponding to the target clothing size; The target three-dimensional clothing model is worn on the three-dimensional human body model to obtain a virtual fitting model.

8. A virtual fitting model generation device, characterized in that: The device comprises: An acquisition module, used for acquiring target shape information, wherein the target shape information includes a two-dimensional target image and target shape description information; wherein the target is a human body or clothing; A communication module, used for inputting the two-dimensional target image and the target appearance description information into a content generation model, and obtaining a plurality of two-dimensional target images output by the content generation model, wherein the plurality of two-dimensional target images are images of the target at different angles; A generating module is used to obtain a three-dimensional target model according to the multiple two-dimensional target images.

9. A virtual fitting model generation device, characterized in that: include: memory and at least one processor; The memory is used to store computer programs; The at least one processor is configured to execute the computer program stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon; When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images