Virtual fitting method and device, readable storage medium and program product
By masking the clothing area of the original model and performing body parameterization processing, combined with 3D body model rendering and preset clothing posture data, accurate virtual try-on images are generated, which solves the problems of poor virtual try-on effect and unadjustable model body shape in the existing technology, and realizes accurate rendering and editing of the model body shape.
Patent Information
- Application Number
- CN202510840477.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-23
- Publication Date
- 2025-09-26
AI Technical Summary
Existing virtual fitting technology cannot achieve the same quality as actual model studio shots, and the model's body data is fixed and cannot be adjusted, resulting in users being unable to view the fitting effects of different body shapes.
By masking the clothing area of the original model, a general model image is obtained, and human body parameterization is performed to obtain the target human body model parameters. Combined with three-dimensional human body model rendering and preset clothing posture data, the generation of virtual try-on images is achieved.
It achieves accurate rendering of human body models during the virtual try-on process, maintains and edits the model's body shape easily and efficiently, and generates virtual try-on images that meet expectations.
Smart Images
Figure CN120707724A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a virtual try-on method, device, computer-readable storage medium, and computer program product. Background Art
[0002] With the development of image processing technology, application technologies for virtual try-on based on diffusion model technology have emerged. This virtual try-on technology eliminates the need for users to hire models for clothing photography at high costs. However, the current technical level cannot achieve the same quality as actual model studio photography. In addition, the actual fitting effect is often unadjustable due to the fixed body data of the model. This makes it impossible for users to view the fitting effect achieved by the virtual try-on function on models with different body shapes. Summary of the Invention
[0003] Based on this, the present application provides a virtual try-on method, apparatus, computer equipment, computer-readable storage medium and computer program product, which can, while using a diffusion model to perform virtual try-on tasks, add the rendering results of the three-dimensional human body model of the original model as new guidance conditions on the basis of the guidance conditions originally input into the human skeleton. By modifying the relevant parameters of the three-dimensional human body model, the original model's body shape can be controlled and maintained to achieve different try-on effects.
[0004] In one aspect, the present application provides a virtual try-on method, comprising:
[0005] By masking the clothing area of the original model, a general model image is obtained;
[0006] Performing human body parameterization processing on the original model to obtain target human body model parameters;
[0007] Performing human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result;
[0008] A virtual try-on is performed based on the general model image, the human body model rendering result and preset clothing posture data to obtain a virtual try-on image.
[0009] In one embodiment, the target human body model parameters include a target body coefficient and a target posture coefficient; performing human body parameterization processing on the original model to obtain the target human body model parameters includes:
[0010] Performing a body parameter analysis on the original model according to a human body model coefficient analysis model to obtain the target body coefficient;
[0011] The original model is subjected to posture parameter analysis according to a human body model coefficient analysis model to obtain the target posture coefficient.
[0012] In one embodiment, performing human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result includes:
[0013] Performing body shape rendering on the three-dimensional human body model of the original model according to the target body shape coefficient of the target human body model parameter to obtain the target human body shape;
[0014] The target human body shape is subjected to posture rendering according to the target posture coefficient of the target human body model parameter to obtain the human body model rendering result.
[0015] In one embodiment, performing a virtual try-on based on the general model image, the human body model rendering result, and preset clothing posture data to obtain a virtual try-on image includes:
[0016] Performing convolution processing on the general model image, the human body model rendering result, and the posture data of the preset clothing posture data according to a preset initialization convolution layer to obtain an initialized rendering model;
[0017] A virtual try-on is performed on the clothing data of the initialized rendering model and the preset clothing posture data according to a preset diffusion model to obtain the virtual try-on image.
[0018] In one embodiment, the diffusion model includes: a clothing diffusion model and a virtual try-on generation model; performing a virtual try-on on the preset clothing posture data and the initialized rendering model according to the preset diffusion model to obtain the virtual try-on image includes:
[0019] Performing clothing fusion diffusion on the clothing data of the initialized rendering model and the preset clothing posture data according to the clothing diffusion model to obtain candidate clothing fitting data;
[0020] A virtual try-on image is generated based on the candidate clothing try-on data, the initialization rendering model and the posture data of the preset clothing posture data according to the virtual try-on generation model to obtain the virtual try-on image.
[0021] In one embodiment, after performing convolution processing on the general model image, the human body model rendering result, and the posture data of the preset clothing posture data according to the preset initialization convolution layer to obtain the initialized rendering model, the method further includes:
[0022] Performing linear mapping according to the target body coefficients of the target human body model parameters to obtain a global body embedding vector;
[0023] A virtual try-on is performed on the clothing data of the initialized rendering model, the global body shape embedding vector, and the preset clothing posture data according to a preset diffusion model to obtain the virtual try-on image.
[0024] In one embodiment, after performing human body parameterization processing on the original model to obtain target human body model parameters, the method further includes:
[0025] Get updated human body model parameters;
[0026] updating the target human body model parameters according to the updated human body model parameters to obtain updated target human body model parameters;
[0027] Performing human body recognition rendering on the three-dimensional human body model of the original model according to the updated target human body model parameters to obtain an updated human body model rendering result;
[0028] A virtual try-on is performed based on the general model image, the updated human body model rendering result and preset clothing posture data to obtain an updated virtual try-on image.
[0029] On the one hand, the present application also provides a virtual try-on device, comprising:
[0030] a mask processing module, configured to obtain a general model image by performing mask processing on the clothing area of the original model;
[0031] A parameterization processing module, configured to perform human body parameterization processing on the original model to obtain target human body model parameters;
[0032] A human body recognition rendering module is used to perform human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result;
[0033] The virtual try-on module is used to perform a virtual try-on based on the general model image, the human body model rendering result and the preset clothing posture data to obtain a virtual try-on image.
[0034] In one aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0035] By masking the clothing area of the original model, a general model image is obtained;
[0036] Performing human body parameterization processing on the original model to obtain target human body model parameters;
[0037] Performing human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result;
[0038] A virtual try-on is performed based on the general model image, the human body model rendering result and preset clothing posture data to obtain a virtual try-on image.
[0039] In one aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0040] By masking the clothing area of the original model, a general model image is obtained;
[0041] Performing human body parameterization processing on the original model to obtain target human body model parameters;
[0042] Performing human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result;
[0043] A virtual try-on is performed based on the general model image, the human body model rendering result and preset clothing posture data to obtain a virtual try-on image.
[0044] In one aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:
[0045] By masking the clothing area of the original model, a general model image is obtained;
[0046] Performing human body parameterization processing on the original model to obtain target human body model parameters;
[0047] Performing human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result;
[0048] A virtual try-on is performed based on the general model image, the human body model rendering result and preset clothing posture data to obtain a virtual try-on image.
[0049] The above-mentioned virtual try-on method, apparatus, computer device, computer-readable storage medium, and computer program product obtain a general model image by masking the clothing area of the original model; perform body parameterization processing on the original model to obtain target body model parameters; perform body recognition rendering on the three-dimensional body model of the original model using the target body model parameters to obtain a body model rendering result; and perform virtual try-on based on the general model image, the body model rendering result, and preset clothing pose data to obtain a virtual try-on image. Therefore, by performing body parameterization processing on the original model to obtain accurate target body model parameters, and performing body recognition rendering on the three-dimensional body model of the original model using the target body model parameters, the body model rendering result can be accurately rendered. At the same time, the target body model parameters, as parameters for accurately reflecting the body model, are easy to control and maintain during the specific rendering process. Therefore, the virtual try-on based on the general model image, the body model parameter body model rendering result, and the preset clothing pose data can be used to perform virtual try-on. The obtained virtual try-on image can simply and efficiently achieve the model's body shape maintenance and editing functions by changing the target body model parameters as a guiding condition during the virtual try-on process. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.
[0051] Figure 1 A diagram illustrating an application environment of a virtual try-on method in one embodiment;
[0052] Figure 2 1 is a flow chart of a virtual try-on method according to an embodiment;
[0053] Figure 3 1 is a flow chart of a virtual try-on method according to an embodiment;
[0054] Figure 4 is a flowchart of a virtual try-on method in another embodiment;
[0055] Figure 5 is a flowchart of a virtual try-on method in another embodiment;
[0056] Figure 6 is a structural block diagram of a virtual try-on device in one embodiment;
[0057] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solutions and beneficial effects of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0059] The virtual try-on method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be integrated with server 104 or placed on a cloud or other network server. Server 104 performs masking on the clothing area of the original model to obtain a general model image; performs human body parameterization on the original model to obtain target human body model parameters; performs human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result; and performs a virtual try-on based on the general model image, the human body model rendering result, and preset clothing posture data to obtain a virtual try-on image.
[0060] Terminal 102 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart car devices, and projectors. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Head-mounted devices may include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, and the like. Server 104 may be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.
[0061] In an exemplary embodiment, Figure 2 As shown, a virtual try-on method is provided, which is applied to Figure 1 The server 104 in the example is used as an example to illustrate the process, including the following steps 202 to 208.
[0062] Step 202 : Obtain a general model image by performing mask processing on the clothing area of the original model.
[0063] Here, the general model image refers to an agnostic-model image.
[0064] In some embodiments, a general model image can be obtained by inputting mask data into the clothing area of the original model. The general model image can be used as a general model image combined with other clothing posture data. The general model image can be obtained quickly and efficiently by masking the clothing area of the original model.
[0065] Step 204: Perform human body parameterization processing on the original model to obtain target human body model parameters.
[0066] Among them, human body parameterization processing refers to parameterizing the body shape and posture of the original model based on SMPL. SMPL is a commonly used three-dimensional human body model, and its full name is Skinned Multi-Person Linear model.
[0067] In some embodiments, the target human body model parameters include a target body shape coefficient and a target posture coefficient; performing human body parameterization processing on the original model to obtain the target human body model parameters includes: performing body parameter analysis on the original model according to the human body model coefficient analysis model to obtain the target body shape coefficient; performing posture parameter analysis on the original model according to the human body model coefficient analysis model to obtain the target posture coefficient.
[0068] In some embodiments, the body parameters of the original model are analyzed based on the human body model coefficient analysis model to obtain a target body coefficient β of the target human body model parameters. The target body coefficient β can be used to control the rendering results of different body shapes, such as fatter or thinner.
[0069] In some embodiments, the original model is subjected to posture parameter analysis based on the human body model coefficient analysis model to obtain a target posture coefficient α of the target human body model parameter. The target posture coefficient α can be used to control rendering results of different postures.
[0070] Step 206 , performing human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result.
[0071] The human body model rendering result may be in densepose format, or may be a depth image and a normal image of the rendered human body model, but is not limited thereto.
[0072] In some embodiments, human body recognition rendering is performed on the three-dimensional human body model of the original model using target human body model parameters to obtain a human body model rendering result, including: body rendering of the three-dimensional human body model of the original model according to the target body coefficient of the target human body model parameters to obtain the target human body body; posture rendering of the target human body body according to the target posture coefficient of the target human body model parameters to obtain the human body model rendering result.
[0073] In some embodiments, the target body coefficient and target posture coefficient of the target human body model parameters can be used for body rendering and posture rendering, respectively, thereby controlling the body posture of the rendered 3D human body model and obtaining a human body model rendering result.
[0074] Step 208 : Perform a virtual try-on based on the general model image, the human body model rendering result, and the preset clothing posture data to obtain a virtual try-on image.
[0075] Among them, the preset clothing posture data may include clothing data and posture data. It should be noted that virtual try-on refers to the process of generating a virtual try-on image based on guiding conditions such as a general model image, a human body model rendering result and preset clothing posture data.
[0076] In some embodiments, a virtual try-on is performed based on a general model image, a human body model rendering result, and preset clothing posture data to obtain a virtual try-on image, including: performing convolution processing on the general model image, the human body model rendering result, and the posture data of the preset clothing posture data according to a preset initialization convolution layer to obtain an initialized rendering model; and performing a virtual try-on on the clothing data of the initialized rendering model and the preset clothing posture data according to a preset diffusion model to obtain a virtual try-on image.
[0077] The preset initialization convolution layer may use init conv, and the convolution processing refers to the concat processing.
[0078] In some embodiments, a rendering data model that completes preliminary data fusion, namely, an initialized rendering model, is obtained by performing convolution processing on the posture data of the general model image, the human body model rendering result, and the preset clothing posture data through a preset initialization convolution layer.
[0079] In some embodiments, the diffusion model includes: a clothing diffusion model and a virtual try-on generation model; performing a virtual try-on on preset clothing posture data and an initialized rendering model according to the preset diffusion model to obtain a virtual try-on image, including: performing clothing fusion diffusion on the clothing data of the initialized rendering model and the preset clothing posture data according to the clothing diffusion model to obtain candidate clothing try-on data; performing a virtual try-on image generation on the posture data of the candidate clothing try-on data, the initialized rendering model and the preset clothing posture data according to the virtual try-on generation model to obtain a virtual try-on image.
[0080] Among them, the clothing diffusion model uses the garmentNet model, and the virtual try-on generation model uses the generateNet model. Both models are implemented using the diffusion model, and the type of the diffusion model is not limited to the SD series or DIT.
[0081] In some embodiments, data fusion is performed on the clothing data of the initialized rendering model and the preset clothing posture data, and then the fused clothing data is input into the clothing diffusion model, and the fused clothing data is diffused through the clothing diffusion model to obtain candidate clothing fitting data.
[0082] Among them, the data fusion process adopts Element-wise-sum processing.
[0083] In some embodiments, virtual try-on images are generated based on the candidate clothing try-on data, the initialization rendering model and the posture data of the preset clothing posture data according to the virtual try-on generation model. The rendered initialization rendering model can be effectively used as one of the input conditions to try on the model of the candidate clothing try-on data, so that the effect of trying on clothes can be achieved while maintaining the figure of the original model and the posture data of the preset clothing posture data, and obtaining accurate and expected virtual try-on images.
[0084] It should be further explained that in some cases the SMPL rendering effect may not accurately match the original model's posture. Therefore, it is necessary to preset the clothing posture data as the model's pose condition image. More often, the rendered human body recognition rendering results are used to control the body shape, while the guidance of the human body recognition rendering results on the posture is weakened. Therefore, in the virtual try-on process, the posture guidance is more responsible for the posture data.
[0085] In some embodiments, after convolution processing is performed on the general model image, the human body model rendering result, and the posture data of the preset clothing posture data according to a preset initialization convolution layer to obtain an initialized rendering model, the method further includes: performing linear mapping according to the body coefficient of the target human body model parameters to obtain a global body embedding vector; and performing virtual try-on on the initialized rendering model, the global body embedding vector, and the clothing data of the preset clothing posture data according to a preset diffusion model to obtain a virtual try-on image.
[0086] In some embodiments, linear mapping is performed based on the target body coefficient of the target human body model parameters, that is, the target body coefficient can be mapped through a linear layer to a dimension that can adapt to the generative model input as a global body embedding vector similar to the time-embedding global vector, and the global body embedding vector is input into a preset diffusion model to enhance its global body control capability. According to the preset diffusion model, the clothing data of the initialized rendering model, the global body embedding vector and the preset clothing posture data are virtually tried on to obtain a virtual try-on image that is closer to the body shape corresponding to the target body coefficient.
[0087] In the above-mentioned virtual try-on method, a general model image is obtained by masking the clothing area of the original model; the original model is subjected to human body parameterization processing to obtain target human body model parameters; the original model's three-dimensional human body model is subjected to human body recognition rendering using the target human body model parameters to obtain a human body model rendering result; and a virtual try-on is performed based on the general model image, the human body model rendering result, and preset clothing posture data to obtain a virtual try-on image. Therefore, by performing human body parameterization processing on the original model to obtain accurate target human body model parameters, and then performing human body recognition rendering on the original model's three-dimensional human body model using the target human body model parameters, a human body model rendering result can be accurately rendered. At the same time, the target human body model parameters, as parameters for accurately reflecting the human body model, are easy to control and maintain during the specific rendering process. Therefore, the virtual try-on is performed based on the general model image, the human body model parameter human body model rendering result, and the preset clothing posture data. The virtual try-on image obtained can simply and efficiently achieve the model's body shape maintenance and editing functions by changing the target human body model parameters as a guiding condition during the virtual try-on process.
[0088] In an exemplary embodiment, Figure 3 As shown, in order to more easily and efficiently implement the model figure preservation and editing functions, the virtual try-on method further includes steps 302 to 308. Among them:
[0089] Step 302: Acquire and update human body model parameters.
[0090] The updated human body model parameters are human body model parameters of other models, that is, the human body model parameters of other models can be used as updated human body model parameters for updating.
[0091] In some embodiments, the updated human body model parameters can be obtained through direct user input, or the image data of other models can be directly input and the updated human body model parameters can be obtained based on the image data of other models, or the updated human body model parameters with default settings can be obtained by sending update instructions, but are not limited to this.
[0092] Step 304 : updating the target human body model parameters according to the updated human body model parameters to obtain updated target human body model parameters.
[0093] The parameter update may be to modify or replace the target human body model parameters.
[0094] In some embodiments, the target human body model parameters are updated according to the updated human body model parameters, that is, the target body coefficient and the target posture coefficient in the target human body model parameters are updated to the coefficients corresponding to the updated human body model parameters, thereby conveniently obtaining the updated target human body model parameters.
[0095] Step 306 : Perform human body recognition rendering on the original model's three-dimensional human body model according to the updated target human body model parameters to obtain an updated human body model rendering result.
[0096] In some embodiments, the three-dimensional human body model of the original model is subjected to human body recognition rendering according to the updated target human body model parameters to obtain an updated human body model rendering result, that is, the updated target human body model parameters are rendered as the corresponding updated human body model rendering result, and used as a new guiding condition for subsequent virtual try-on steps.
[0097] Step 308 : Perform a virtual try-on based on the general model image, the updated human body model rendering result, and the preset clothing posture data to obtain an updated virtual try-on image.
[0098] In some embodiments, by using the updated human body model rendering result as a guiding condition, a virtual try-on is performed based on the general model image, the updated human body model rendering result and the preset clothing posture data to obtain an updated virtual try-on image, which can easily and efficiently realize the model figure maintenance and editing functions in the updated virtual try-on image.
[0099] In this embodiment, by real-time input or providing update instructions, the virtual try-on image can be rendered and updated in real time, thereby simply and efficiently achieving the model's figure maintenance and editing functions by changing the guidance conditions.
[0100] In order to understand this application more clearly, Figure 4 The details are as follows:
[0101] First, if Figure 4 As shown, Figure 4 The following illustrates the flow of specific data in a simple and complete embodiment of the present invention, as well as the corresponding results generated in each step, and finally the process of generating a virtual try-on image based on the results of each step. The specific steps and corresponding results are defined and described in the above embodiments, so they will not be detailed here.
[0102] Secondly, in order to explain some implementation details of the specific embodiment in more detail, Figure 5 For further explanation, such as Figure 5 As shown, Figure 5 A more detailed implementation step is shown. Figure 5 E(agno) in the equation refers to the general model image. Figure 5E (SMPL-densepose|depth|normal) refers to the rendering result of the human body model, E (pose) is the pose data in the preset clothing pose data, which can also be called the human skeleton map, E (cloth) is the clothing data in the preset clothing pose data, Zt is a random noise image, Init conv refers to the preset initialization convolution layer, and embeddinglayer refers to the embedding layer.
[0103] In addition, Figure 5 The main idea of this scheme is clearly demonstrated in the paper. By masking the clothing area of the original model, a general model image is obtained. The original model is parameterized to obtain the target human body model parameters. The three-dimensional human body model of the original model is rendered by human body recognition using the target human body model parameters to obtain the human body model rendering result. A virtual try-on is performed based on the general model image, the human body model rendering result and the preset clothing posture data to obtain a virtual try-on image. At the same time, it is demonstrated how to perform linear mapping of the body coefficient to the embedding layer to obtain a global body embedding vector similar to the time-embedding global vector, and apply it to the virtual try-on generation model. It also shows that the process of replacing the human body model parameters of the original model for other models in this scheme is very simple. The result of the virtual try-on can be efficiently changed by replacing or modifying the body coefficient and posture coefficient of the target human body model parameters.
[0104] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.
[0105] Based on the same inventive concept, the present application also provides a virtual try-on device for implementing the aforementioned virtual try-on method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more virtual try-on device embodiments provided below can be found in the above-mentioned limitations of the virtual try-on method and will not be further elaborated here.
[0106] In an exemplary embodiment, Figure 6As shown, a virtual try-on device is provided, comprising: a mask processing module 601, a parameterized processing module 602, a human body recognition rendering module 603 and a virtual try-on module 604, wherein:
[0107] The mask processing module 601 is used to obtain a general model image by performing mask processing on the clothing area of the original model;
[0108] The parameterization processing module 602 is used to perform human body parameterization processing on the original model to obtain target human body model parameters;
[0109] The human body recognition rendering module 603 is used to perform human body recognition rendering on the original model's three-dimensional human body model using the target human body model parameters to obtain a human body model rendering result;
[0110] The virtual try-on module 604 is used to perform a virtual try-on based on the general model image, the human body model rendering result and the preset clothing posture data to obtain a virtual try-on image.
[0111] In some embodiments, the target human body model parameters include a target body coefficient and a target posture coefficient; the parameterization processing module 602 is further used to analyze the body parameters of the original model according to the human body model coefficient analysis model to obtain the target body coefficient; and to analyze the posture parameters of the original model according to the human body model coefficient analysis model to obtain the target posture coefficient.
[0112] In some embodiments, the human body recognition rendering module 603 is also used to perform body rendering on the three-dimensional human body model of the original model according to the target body coefficient of the target human body model parameters to obtain the target human body body; and perform posture rendering on the target human body body according to the target posture coefficient of the target human body model parameters to obtain the human body model rendering result.
[0113] In some embodiments, the virtual try-on module 604 is further configured to perform convolution processing on the general model image, the human body model rendering result, and the posture data of the preset clothing posture data according to a preset initialization convolution layer to obtain an initialized rendering model;
[0114] A virtual try-on is performed on the clothing data of the initialized rendering model and the preset clothing posture data according to a preset diffusion model to obtain a virtual try-on image.
[0115] In some embodiments, the diffusion model includes: a clothing diffusion model and a virtual try-on generation model; the virtual try-on module 604 is also used to perform clothing fusion diffusion on the clothing data of the initialization rendering model and the preset clothing posture data according to the clothing diffusion model to obtain candidate clothing try-on data; and generate a virtual try-on image on the posture data of the candidate clothing try-on data, the initialization rendering model and the preset clothing posture data according to the virtual try-on generation model to obtain a virtual try-on image.
[0116] In some embodiments, after convolution processing is performed on the general model image, the human body model rendering result, and the posture data of the preset clothing posture data according to the preset initialization convolution layer to obtain the initialized rendering model, the virtual try-on module 604 is further used to perform linear mapping according to the target body coefficient of the target human body model parameters to obtain a global body embedding vector; and perform virtual try-on on the initialized rendering model, the global body embedding vector, and the clothing data of the preset clothing posture data according to the preset diffusion model to obtain a virtual try-on image.
[0117] In some embodiments, after performing human body parameterization processing on the original model to obtain target human body model parameters, the device also includes: updating a virtual try-on module for obtaining updated human body model parameters; updating the target human body model parameters according to the updated human body model parameters to obtain updated target human body model parameters; performing human body recognition rendering on the three-dimensional human body model of the original model according to the updated target human body model parameters to obtain updated human body model rendering results; performing a virtual try-on based on the general model image, the updated human body model rendering results and the preset clothing posture data to obtain an updated virtual try-on image.
[0118] In the above-mentioned virtual try-on device, a general model image is obtained by masking the clothing area of the original model; the original model is subjected to human body parameterization processing to obtain target human body model parameters; the original model's three-dimensional human body model is subjected to human body recognition rendering using the target human body model parameters to obtain a human body model rendering result; and a virtual try-on is performed based on the general model image, the human body model rendering result, and preset clothing posture data to obtain a virtual try-on image. Therefore, by performing human body parameterization processing on the original model to obtain accurate target human body model parameters, and then performing human body recognition rendering on the original model's three-dimensional human body model using the target human body model parameters, a human body model rendering result can be accurately rendered. At the same time, the target human body model parameters, as parameters for accurately reflecting the human body model, are easy to control and maintain during the specific rendering process. Therefore, the virtual try-on is performed based on the general model image, the human body model parameter human body model rendering result, and the preset clothing posture data. The virtual try-on image obtained can simply and efficiently achieve the model's body shape maintenance and editing functions by changing the target human body model parameters as a guiding condition during the virtual try-on process.
[0119] Each module in the aforementioned virtual try-on device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a computer device's memory in the form of software, so that the processor can call and execute the corresponding operations of each module.
[0120] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store diffusion model data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, mobile cellular networks, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a virtual try-on method. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.
[0121] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.
[0122] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.
[0123] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0124] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0125] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0126] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.
[0127] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0128] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.
Claims
1. A virtual try-on method, characterized in that: The method comprises: By masking the clothing area of the original model, a general model image is obtained; Performing human body parameterization processing on the original model to obtain target human body model parameters; Performing human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result; A virtual try-on is performed based on the general model image, the human body model rendering result and preset clothing posture data to obtain a virtual try-on image.
2. The method according to claim 1, characterized in that The target human body model parameters include a target body coefficient and a target posture coefficient; the human body parameterization processing of the original model to obtain the target human body model parameters includes: Performing a body parameter analysis on the original model according to a human body model coefficient analysis model to obtain the target body coefficient; The original model is subjected to posture parameter analysis according to a human body model coefficient analysis model to obtain the target posture coefficient.
3. The method according to claim 1, characterized in that The step of performing human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result includes: Performing body shape rendering on the three-dimensional human body model of the original model according to the target body shape coefficient of the target human body model parameter to obtain the target human body shape; The target human body shape is subjected to posture rendering according to the target posture coefficient of the target human body model parameter to obtain the human body model rendering result.
4. The method according to claim 1, wherein The performing virtual try-on based on the general model image, the human body model rendering result and the preset clothing posture data to obtain a virtual try-on image includes: Performing convolution processing on the general model image, the human body model rendering result, and the posture data of the preset clothing posture data according to a preset initialization convolution layer to obtain an initialized rendering model; A virtual try-on is performed on the clothing data of the initialized rendering model and the preset clothing posture data according to a preset diffusion model to obtain the virtual try-on image.
5. The method according to claim 4, characterized in that The diffusion model includes: a clothing diffusion model and a virtual try-on generation model; performing a virtual try-on on the preset clothing posture data and the initialized rendering model according to the preset diffusion model to obtain the virtual try-on image includes: Performing clothing fusion diffusion on the clothing data of the initialized rendering model and the preset clothing posture data according to the clothing diffusion model to obtain candidate clothing fitting data; A virtual try-on image is generated based on the candidate clothing try-on data, the initialization rendering model and the posture data of the preset clothing posture data according to the virtual try-on generation model to obtain the virtual try-on image.
6. The method according to claim 4, characterized in that After performing convolution processing on the general model image, the human body model rendering result, and the posture data of the preset clothing posture data according to the preset initialization convolution layer to obtain the initialized rendering model, the method further includes: Performing linear mapping according to the target body coefficients of the target human body model parameters to obtain a global body embedding vector; A virtual try-on is performed on the clothing data of the initialized rendering model, the global body shape embedding vector, and the preset clothing posture data according to a preset diffusion model to obtain the virtual try-on image.
7. The method according to claim 1, characterized in that After performing human body parameterization processing on the original model to obtain target human body model parameters, the method further includes: Get updated human body model parameters; updating the target human body model parameters according to the updated human body model parameters to obtain updated target human body model parameters; Performing human body recognition rendering on the three-dimensional human body model of the original model according to the updated target human body model parameters to obtain an updated human body model rendering result; A virtual try-on is performed based on the general model image, the updated human body model rendering result and preset clothing posture data to obtain an updated virtual try-on image.
8. A virtual try-on device, characterized in that: The device comprises: a mask processing module, configured to obtain a general model image by performing mask processing on the clothing area of the original model; A parameterization processing module, configured to perform human body parameterization processing on the original model to obtain target human body model parameters; A human body recognition rendering module is used to perform human body recognition rendering on the three-dimensional human body model of the original model using the target human body model parameters to obtain a human body model rendering result; The virtual try-on module is used to perform a virtual try-on based on the general model image, the human body model rendering result and the preset clothing posture data to obtain a virtual try-on image.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.