Virtual fitting method, related apparatuses and communication system

Abstract

The application provides a virtual fitting method, related device and communication system. A mobile terminal can send 3D image data, 3D digital clothing data and action data of a user to a server in response to the user selecting clothing and a fitting action to perform virtual fitting. The server can calculate clothing deformation information after the human body and the clothing collide under the corresponding fitting action through physical simulation technology to obtain simulation data of the clothing. The mobile terminal can render the fitting effect of the user's 3D image wearing the clothing to perform the fitting action according to the simulation data sent by the server. The above fitting effect can realistically present the style, fabric and other clothing information of the clothing, and the deformation information of details such as wrinkles caused by the collision between the 3D image and the digital clothing. This can help the user to understand the fitting effect of the clothing before placing an order to purchase in an online clothing market, reduce the situation of buying unsuitable clothing online, and reduce the return rate of clothing e-commerce.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, and in particular to a virtual fitting method, related devices and a communication system. BACKGROUND

[0002] With the development of online clothing market, more and more users purchase clothes online through clothing e-commerce. However, users cannot try on clothes before purchasing them online, and often need to return the clothes purchased online due to unsuitability. This results in poor user shopping experience and high return rate of clothing e-commerce. SUMMARY

[0003] The present application provides a virtual fitting method, related devices and a communication system. The electronic device can render a fitting effect of a 3D image of a user wearing a selected clothing according to the body data of the user and the selected clothing and fitting action. The fitting effect can realistically present the style, fabric and other clothing information of the clothing, and the deformation information of details such as wrinkles caused by the collision between the clothing and the 3D image. In the case of different fitting actions of the 3D image, the collision between the clothing and the 3D image can produce different deformations. The electronic device can render different fitting effects corresponding to different fitting actions of the 3D image, and present the dynamic changes of the clothing in the process of different fitting actions of the 3D image to the user.

[0004] In a first aspect, the present application provides a virtual fitting method. The electronic device receives a first operation for selecting a first clothing for virtual fitting. In response to the first operation, the electronic device sends first body data of a first 3D image, first action data of a first action sequence and first clothing data of the first clothing to a server, wherein the first action sequence includes a first action and a second action. The electronic device receives first simulation data of the first clothing from the server. The electronic device renders a first set of fitting effects according to the first simulation data, wherein the first set of fitting effects includes a first fitting effect and a second fitting effect, the first fitting effect is used to show the effect of the first 3D image wearing the first clothing to perform the first action, and the second fitting effect is used to show the effect of the first 3D image wearing the first clothing to perform the second action, and the first fitting effect and the second fitting effect are different.

[0005] The first simulation data can be obtained by the server through fabric simulation according to the first body data, the first action data and the first clothing data.

[0006] The first operation can refer to the operation of the virtual fitting control 512 shown in the subsequent embodiments of the present application. Figure 5B

[0007] ​The method can realize providing a virtual fitting function on a mobile terminal through end-cloud cooperation, and present a high-precision dynamic virtual fitting effect. In this way, when a user shops in an online clothing market, the user can perform virtual fitting and view a 3D image matching the user's body shape wearing a fitting effect of a garment. The fitting effect can truly reflect the effect of the user wearing the corresponding garment. This can help the user understand the fitting effect of the garment before placing an order in the online clothing market, reduce the situation of buying an unsuitable garment in the online clothing market, and reduce the return rate of the clothing e-commerce.

[0008] In combination with the first aspect, in some embodiments, the first body data includes first body mesh data of the first 3D image, the first action data includes first skeleton data of the first action and second skeleton data of the second action, and the first garment data includes garment mesh data and garment parameter information of the first garment, the garment parameter information including one or more of the following: a stretching coefficient, a bending coefficient, and a friction coefficient of the first garment.

[0009] Alternatively, the first body data includes body parameters of the first 3D image, the body parameters being used to generate the first body mesh data, the first action data includes identification information of the first action and the second action, and the first garment data includes identification information of the first garment.

[0010] As known from the above embodiments, the electronic device can reduce the data transmission amount, save the transmission bandwidth between the electronic device and the server, and improve the efficiency of virtual fitting by sending the body parameters, the identification information of the fitting actions (such as the first action and the second action), and the identification information of the first garment to the server.

[0011] In combination with the first aspect, in some embodiments, the electronic device can determine that the first garment can be virtually fitted; the electronic device provides a first control, the first control being used to select the first garment for virtual fitting; and the first operation can be an operation on the first control.

[0012] In combination with the first aspect, in some embodiments, before the electronic device sends the first body data of the first 3D image, the first action data of the first action sequence, and the first garment data of the first garment to the server, the electronic device can also receive an operation of selecting the first 3D image and / or the first action sequence.

[0013] From the above embodiments, it can be seen that the user can select to use his or her own or another person's 3D image for virtual fitting. During the virtual fitting process, the user can adjust the virtual fitting clothes and the fitting actions performed by the 3D image. The electronic device 100 can render the fitting effect of the 3D image wearing the clothes and performing the fitting actions according to the clothes and the fitting actions selected by the user, to help the user accurately determine whether the clothes are suitable for the user corresponding to the 3D image. This can effectively reduce the situation that the user buys unsuitable clothes online, and reduce the return rate of clothes e-commerce.

[0014] In combination with the first aspect, in some embodiments, the first set of fitting effects can be presented by playing a first video. The electronic device receives a second operation for the first video, the second operation being used to play the first video, or being used to pause playing the first video, or being used to fast forward the first video, or being used to fast backward the first video; and the electronic device controls playing of the first video according to the second operation.

[0015] During the playing of the first video, the first 3D image can wear the first clothes to be fitted and perform the actions in the first action sequence in sequence. In this way, the user can view the dynamic virtual fitting effect. Moreover, the user can control the electronic device to pause playing the fitting effect, view and adjust the playing progress of the fitting effect.

[0016] In combination with the first aspect, in some embodiments, the electronic device receives a third operation, the third operation being used to select a second clothes for virtual fitting; in response to the third operation, the electronic device sends second clothes data of the second clothes and first hierarchical information to the server, the first hierarchical information being used to indicate the wearing hierarchy of the first clothes and the second clothes; the electronic device receives second simulation data from the server; and the electronic device renders a second set of fitting effects according to the second simulation data, the second set of fitting effects including a third fitting effect and a fourth fitting effect, the third fitting effect being used to represent the effect of the first 3D image wearing the first clothes and the second clothes to perform a first action, and the fourth fitting effect being used to represent the effect of the first 3D image wearing the first clothes and the second clothes to perform a second action.

[0017] From the above embodiments, it can be seen that the user can select to virtually fit a single piece of clothes, and can also select to fit a plurality of clothes in combination to view the fitting effect of the combination of the plurality of clothes.

[0018] In some embodiments, in combination with the first aspect, the electronic device receives a fourth operation for selecting a third garment to replace the first garment for virtual fitting; in response to the fourth operation, the electronic device sends third garment data of the third garment to the server; the electronic device receives third simulation data of the third garment from the server; and the electronic device renders a third set of fitting effects based on the third simulation data, the third set of fitting effects including a fifth fitting effect and a sixth fitting effect, the fifth fitting effect representing an effect of the first 3D avatar performing the first action in the third garment, and the sixth fitting effect representing an effect of the first 3D avatar performing the second action in the third garment.

[0019] As can be seen from the above embodiments, in the process of virtual fitting, the user can select one or more garments to replace the garments already tried on by the 3D avatar. For example, the user can adjust the size, color, or other styles of the garments tried on by the 3D avatar.

[0020] In some embodiments, in combination with the first aspect, the electronic device receives a fifth operation for changing the first action sequence to a second action sequence, the second action sequence including a third action and a fourth action; in response to the fifth operation, the electronic device sends second action data of the second action sequence to the server; the electronic device receives fourth simulation data of the first garment from the server; and the electronic device renders a fourth set of fitting effects based on the fourth simulation data, the fourth set of fitting effects including a seventh fitting effect and an eighth fitting effect, the seventh fitting effect representing an effect of the first 3D avatar performing the third action in the first garment, and the eighth fitting effect representing an effect of the first 3D avatar performing the fourth action in the first garment.

[0021] In some embodiments, in combination with the first aspect, the electronic device receives a sixth operation for selecting a fitting scene as a first scene; in response to the sixth operation, the electronic device obtains first scene data of the first scene; and the electronic device renders a first set of fitting effects based on the first scene data and the first simulation data, the first set of fitting effects including a first fitting effect and a second fitting effect, the first fitting effect representing an effect of the first 3D avatar performing the first action in the first garment in the first scene, and the second fitting effect representing an effect of the first 3D avatar performing the second action in the first garment in the first scene.

[0022] As can be seen from the above embodiments, in the process of virtual fitting, the user can set a fitting scene, such as a T-stage, a stage, a square, a grassland, a beach, a house, etc. The electronic device can render fitting effects in the fitting scene. Providing one or more scene options in the virtual fitting scene can increase the playability of virtual fitting. The user can experience virtual fitting effects in different scenes.

[0023] In some embodiments of the first aspect, the electronic device obtains first body size information, the first body size information including one or more of the following: height, shoulder width, bust, waist, hip, arm circumference, arm length, neck circumference, abdominal circumference, crotch circumference, thigh circumference, calf circumference, leg length; and the electronic device obtains first human mesh data of the first 3D avatar based on the first body size information.

[0024] Alternatively, the electronic device obtains one or more images, the one or more images showing a human body corresponding to the first 3D avatar; and the electronic device obtains the first human mesh data of the first 3D avatar based on the one or more images.

[0025] In a second aspect, the present application provides a virtual fitting method. The method can be executed by a processing module in an electronic device. The processing module can be a digital garment simulation SDK. The processing module can be configured to: obtain first human data of a first 3D avatar, first action data of a first action sequence, and first garment data of a first garment, wherein the first action sequence includes a first action and a second action; send the first human data, the first action data, and the first garment data to a server; receive first simulation data of the first garment from the server; and render a first set of fitting effects by a 3D rendering module based on the first simulation data, the first set of fitting effects including a first fitting effect and a second fitting effect, the first fitting effect showing the first 3D avatar wearing the first garment performing the first action, and the second fitting effect showing the first 3D avatar wearing the first garment performing the second action, the first fitting effect being different from the second fitting effect.

[0026] The first simulation data can be obtained by the server based on the first human data, the first action data, and the first garment data.

[0027] The above method can be implemented by end-cloud collaboration to provide a virtual fitting function on a mobile terminal and present a high-precision dynamic virtual fitting effect. In this way, when a user shops in an online clothing market, the user can perform virtual fitting and view a 3D avatar wearing a garment that matches the user's body type. The above fitting effect can realistically reflect the effect of the user wearing the corresponding garment. This can help the user to understand the fitting effect of the garment before placing an order in an online clothing market, reduce the situation of buying unsuitable garments in an online clothing market, and reduce the return rate of clothing e-commerce.

[0028] In some embodiments, the first body data comprises first body mesh data of the first 3D avatar, the first action data comprises first skeleton data of the first action and second skeleton data of the second action, and the first clothing data comprises clothing mesh data and clothing parameter information of the first clothing, the clothing parameter information comprising one or more of the following: a stretch coefficient, a bending coefficient, and a friction coefficient of the first clothing.

[0029] Alternatively, the first body data comprises body parameters of the first 3D avatar, the body parameters being used to generate the first body mesh data, the first action data comprises identification information of the first action and the second action, and the first clothing data comprises identification information of the first clothing.

[0030] As can be seen from the above embodiments, the electronic device can reduce the data transmission amount by sending the body parameters, the identification information of the fitting actions (e.g., the first action and the second action), and the identification information of the first clothing to the server, save the transmission bandwidth between the electronic device and the server, and improve the efficiency of virtual fitting.

[0031] In some embodiments, the processing module can be configured to: pre-process the first simulation data to obtain pre-processed data, the pre-processing comprising one or more of the following: normal calculation on the first simulation data, and data synchronization on the first simulation data and the first body data of the first action sequence of the first 3D avatar; and send the pre-processed data to the 3D rendering module to render the first set of fitting effects.

[0032] The first body data of the first 3D avatar performing the first action sequence can comprise body mesh data corresponding to each action included in the first action sequence of the first 3D avatar. The body mesh data corresponding to different actions of the 3D avatar is different.

[0033] In some embodiments, the processing module can be configured to: obtain second clothing data and first hierarchical information of a second clothing, the first hierarchical information being used to indicate the wearing order of the first clothing and the second clothing; send the second clothing data and the first hierarchical information to the server; receive second simulation data from the server; and send the second simulation data to the 3D rendering module to render a second set of fitting effects, the second set of fitting effects comprising a third fitting effect and a fourth fitting effect, the third fitting effect being used to represent the effect of the first 3D avatar wearing the first clothing and the second clothing and performing the first action, and the fourth fitting effect being used to represent the effect of the first 3D avatar wearing the first clothing and the second clothing and performing the second action.

[0034] As can be seen from the above embodiments, the user can select single clothing virtual fitting, and also can select multi-clothing combination to view the fitting effect of the multi-clothing combination.

[0035] In a third aspect, the present application provides a virtual fitting method. The method can be applied to a communication system. The communication system can include an electronic device and a server. The electronic device receives a first operation for selecting a first garment for virtual fitting. In response to the first operation, the electronic device sends first body data of a first 3D image, first action data of a first action sequence, and first garment data of the first garment to the server, wherein the first action sequence includes a first action and a second action. The server determines first simulation data of the first garment according to the first body data, the first action data, and the first garment data, and sends the first simulation data to the electronic device. The electronic device renders a first set of fitting effects according to the first simulation data, wherein the first set of fitting effects includes a first fitting effect and a second fitting effect, the first fitting effect is used to show the effect of the first 3D image wearing the first garment to perform the first action, and the second fitting effect is used to show the effect of the first 3D image wearing the first garment to perform the second action. The first fitting effect and the second fitting effect are different.

[0036] The above method can be realized by end-cloud cooperation to provide virtual fitting function on a mobile terminal and present high-precision dynamic virtual fitting effect. In this way, when a user shops in an online clothing market, the user can perform virtual fitting and view the fitting effect of the 3D image wearing the garment that matches the user's body type. The above fitting effect can truly reflect the effect of the user wearing the corresponding garment. This can help the user to understand the fitting effect of the garment before placing an order in the online clothing market, reduce the situation of buying unsuitable garments in the online clothing market, and reduce the return rate of the clothing e-commerce.

[0037] In combination with the third aspect, in some embodiments, the first body data includes first body mesh data of the first 3D image, the first action data includes first skeleton data of the first action and second skeleton data of the second action, and the first garment data includes garment mesh data and garment parameter information of the first garment, wherein the garment parameter information includes one or more of the following: a stretch coefficient, a bending coefficient, and a friction coefficient of the first garment.

[0038] Alternatively, the first body data includes body parameters of the first 3D image, the body parameters are used to generate the first body mesh data, the first action data includes identification information of the first action and the second action, and the first garment data includes identification information of the first garment.

[0039] As can be seen from the above embodiments, the electronic device can reduce the data transmission amount, save the transmission bandwidth between the electronic device and the server, and improve the efficiency of virtual fitting by sending the body parameters, the identification information of the fitting actions (such as the first action and the second action), and the identification information of the first garment to the server.

[0040] In some embodiments, the electronic device can determine that the first garment can be virtually tried on; the electronic device can provide a first control for selecting the first garment for virtual try-on; and the first operation can be an operation on the first control.

[0041] In some embodiments, before the electronic device sends the first human body data of the first 3D avatar, the first action data of the first action sequence, and the first garment data of the first garment to the server, the electronic device can receive an operation of selecting the first 3D avatar and / or the first action sequence.

[0042] As can be seen from the above embodiments, the user can select to use his or her own 3D avatar for virtual try-on. During the virtual try-on, the user can adjust the garment for virtual try-on and the try-on action performed by the 3D avatar. The electronic device 100 can render the try-on effect of the 3D avatar wearing the garment and performing the try-on action according to the garment and the try-on action selected by the user, to help the user accurately determine whether the garment is suitable for the user corresponding to the 3D avatar. This can effectively reduce the situation that the user buys an unsuitable garment online, and reduce the return rate of the garment e-commerce.

[0043] In some embodiments, the first group of try-on effects can be presented by a first video playback. The electronic device receives a second operation on the first video, the second operation being used to play the first video, or to pause playing the first video, or to fast forward the first video, or to fast backward the first video; and the electronic device controls the playing of the first video according to the second operation.

[0044] During the above first video playback, the first 3D avatar can wear the first garment for virtual try-on and perform the actions in the first action sequence in sequence. In this way, the user can view the dynamic virtual try-on effect. Moreover, the user can control the electronic device to pause the playing of the try-on effect, view and adjust the playing progress of the try-on effect.

[0045] In some embodiments, the electronic device receives a third operation, the third operation being used to select a second garment for virtual try-on; in response to the third operation, the electronic device sends second garment data of the second garment and first hierarchical information to the server, the first hierarchical information being used to indicate the layering order of the first garment and the second garment; the server determines second simulation data in combination with the second garment data and the first hierarchical information, and sends the second simulation data to the electronic device; and the electronic device renders a second group of try-on effects according to the second simulation data, the second group of try-on effects including a third try-on effect and a fourth try-on effect, the third try-on effect being used to represent the effect of the first 3D avatar wearing the first garment and the second garment to perform a first action, and the fourth try-on effect being used to represent the effect of the first 3D avatar wearing the first garment and the second garment to perform a second action.

[0046] From the above embodiments, it can be seen that the user can select single-piece clothing virtual fitting, and can also select multi-piece clothing collocation to view the fitting effect of the multi-piece clothing collocation.

[0047] In combination with the third aspect, in some embodiments, the electronic device receives a fourth operation for selecting a third piece of clothing to replace the first piece of clothing for virtual fitting; in response to the fourth operation, the electronic device sends third clothing data of the third piece of clothing to the server; the server determines third simulation data of the third piece of clothing according to the first human body data, the first action data, and the third clothing data, and sends the third simulation data to the electronic device; and the electronic device renders a third set of fitting effects according to the third simulation data, the third set of fitting effects including a fifth fitting effect and a sixth fitting effect, the fifth fitting effect being used to represent the effect of the first 3D image wearing the third piece of clothing to perform the first action, and the sixth fitting effect being used to represent the effect of the first 3D image wearing the third piece of clothing to perform the second action.

[0048] From the above embodiments, it can be seen that in the process of virtual fitting, the user can select one or more pieces of clothing to replace the clothing that the 3D image has tried on. For example, the user can adjust the size and color of the clothing worn by the 3D image, or adjust the clothing worn by the 3D image to other styles of clothing.

[0049] In combination with the third aspect, in some embodiments, the electronic device receives a fifth operation for changing the first action sequence to a second action sequence, the second action sequence including a third action and a fourth action; in response to the fifth operation, the electronic device sends second action data of the second action sequence to the server; the server determines fourth simulation data of the first piece of clothing according to the first human body data, the second action data, and the first clothing data, and sends the fourth simulation data to the electronic device; and the electronic device renders a fourth set of fitting effects according to the fourth simulation data, the fourth set of fitting effects including a seventh fitting effect and an eighth fitting effect, the seventh fitting effect being used to represent the effect of the first 3D image wearing the first piece of clothing to perform the third action, and the eighth fitting effect being used to represent the effect of the first 3D image wearing the first piece of clothing to perform the fourth action.

[0050] In combination with the third aspect, in some embodiments, the electronic device receives a sixth operation for selecting a fitting scene as a first scene; in response to the sixth operation, the electronic device obtains first scene data of the first scene; and the electronic device renders a first set of fitting effects according to the first scene data and the first simulation data, the first set of fitting effects including a first fitting effect and a second fitting effect, the first fitting effect being used to represent the effect of the first 3D image wearing the first piece of clothing to perform the first action in the first scene, and the second fitting effect being used to represent the effect of the first 3D image wearing the first piece of clothing to perform the second action in the first scene.

[0051] From the above embodiments, in the process of virtual fitting, the user can set a fitting scene, for example, a T-stage, a stage, a square, a lawn, a beach, a room, and the like. The electronic device can render a fitting effect under the fitting scene. Providing an option of one or more scenes in the virtual fitting scene can increase the playability of virtual fitting. The user can experience the virtual fitting effect under different scenes.

[0052] In combination with the third aspect, in some embodiments, the electronic device obtains first body size information, the first body size information including one or more of the following: height, shoulder width, bust, waist, hip, arm circumference, arm length, neck circumference, abdominal circumference, crotch circumference, thigh circumference, calf circumference, leg length; the electronic device obtains first human mesh data of the first 3D image according to the first body size information.

[0053] Alternatively, the electronic device obtains one or more images, and the one or more images display a human body corresponding to the first 3D image; the electronic device obtains first human mesh data of the first 3D image according to the one or more images.

[0054] In combination with the third aspect, in some embodiments, the server can obtain first human mesh data of the first 3D image according to the first human data, and obtain first skeleton data of the first action and skeleton data of the second action according to the first action data; the server determines second human mesh data of the first 3D image under the first action and third human mesh data of the first 3D image under the second action by using the first human mesh data, the first skeleton data, and the second skeleton data; the server determines the first simulation data according to the second human mesh data, the third human mesh data, and the first clothing data. The first simulation data can include simulation data determined by the second human mesh data and the first clothing data under the collision of the first clothing and the first 3D image performing the first action, and simulation data determined by the third human mesh data and the first clothing data under the collision of the first clothing and the first 3D image performing the second action.

[0055] In combination with the third aspect, in some embodiments, the server stores the first simulation data in a database; the server finds the first simulation data from the database according to the first human data, the first action data, and the first clothing data.

[0056] From the above embodiments, it can be seen that the server can store simulation data of the clothes obtained by the cloth simulation. When the same or similar 3D image needs to be dressed in the same clothes for the same fitting action for cloth simulation again, the server can obtain the simulation data of the clothes from the database without the specific calculation process of the cloth simulation again. In this way, the server can reduce the repeated cloth simulation process, save the computing resources of the server, and improve the concurrent capability of the server side.

[0057] In a fourth aspect, the present application provides an electronic device. The electronic device can include a communication device, a memory and a processor. The communication device can be used for communication between the electronic device and the server. The memory can be used to store a computer program. The processor can be used to call the computer program, so that the electronic device executes any possible implementation method of the first aspect.

[0058] In a fifth aspect, the present application provides a communication system. The communication system can include an electronic device and a server. The electronic device can be used to receive a first operation, the first operation being used to select a first clothes for virtual fitting; the electronic device can also be used to send first human body data of a first 3D image, first action data of a first action sequence and first clothes data of a first clothes to the server in response to the first operation, wherein the first action sequence includes a first action and a second action; the server can be used to determine first simulation data of the first clothes according to the first human body data, the first action data and the first clothes data, and send the first simulation data to the electronic device; the electronic device can be used to render a first set of fitting effects according to the first simulation data, the first set of fitting effects including a first fitting effect and a second fitting effect, the first fitting effect being used to show the effect of the first 3D image wearing the first clothes for the first action, and the second fitting effect being used to show the effect of the first 3D image wearing the first clothes for the second action, the first fitting effect being different from the second fitting effect.

[0059] In a sixth aspect, the present application provides a computer readable storage medium, including instructions, when the instructions are executed on an electronic device, the electronic device executes any possible implementation method of the first aspect.

[0060] In a seventh aspect, the present application provides a computer program product, which can include computer instructions, when the computer instructions are executed on an electronic device, the electronic device executes any possible implementation method of the first aspect.

[0061] In an eighth aspect, the present application provides a chip applied to an electronic device, the chip including one or more processors, the processor being used to call computer instructions to make the electronic device execute any possible implementation method of the first aspect.

[0062] It can be understood that the electronic device provided in the fourth aspect, the communication system provided in the fifth aspect, the computer readable storage medium provided in the sixth aspect, the computer program product provided in the seventh aspect, and the chip provided in the eighth aspect are all used to execute the method provided in the embodiments of the present application. Therefore, the beneficial effects achieved thereby can refer to the beneficial effects in the corresponding method, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 is a structural diagram of a communication system 10 provided by an embodiment of the present application;

[0064] Figure 2A is a structural diagram of an electronic device 100 provided by an embodiment of the present application;

[0065] Figure 2B is a software structural block diagram of an electronic device 100 provided by an embodiment of the present application;

[0066] Figure 3 is a structural diagram of a server 200 provided by an embodiment of the present application;

[0067] Figures 4A-4E is a schematic diagram of some scenes for creating 3D images provided by an embodiment of the present application;

[0068] Figures 5A-5H is a schematic diagram of some scenes for virtual fitting provided by an embodiment of the present application;

[0069] Figure 6 is a flowchart of a virtual fitting method provided by an embodiment of the present application;

[0070] Figure 7 is a structural diagram of another communication system 70 provided by an embodiment of the present application;

[0071] Figure 8 is a structural diagram of another communication system 80 provided by an embodiment of the present application;

[0072] Figure 9 is a structural diagram of another communication system 90 provided by an embodiment of the present application. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing the specific embodiments and are not intended to be limiting on the present application. As used in the specification and the appended claims of the present application, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that “at least one” and “one or more” as used in the embodiments herein indicates one or two or more (including two), unless otherwise indicated. The term “and / or” is used to describe the association relationship of the associated objects, which means that there can be three relationships; for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character “ / ” generally represents an “or” relationship between the associated objects.

[0074] In the present specification, the reference to “one embodiment” or “some embodiments” or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases “in one embodiment”, “in some embodiments”, “in other embodiments”, “in additional embodiments” and so on, in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise indicated. The terms “comprising”, “including”, “having” and their variants mean “including but not limited to”, unless otherwise indicated. The term “connected” includes both direct and indirect connections, unless otherwise indicated. “First”, “second”, etc. are used only for descriptive purposes and should not be construed as indicating or implying relative importance or an indicated number of technical features.

[0075] In the embodiments of the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design described in the embodiments of the present application as “exemplary” or “for example” should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of “exemplary” or “for example” is intended to present concepts in a concrete manner.

[0076] The term "user interface (UI)" in the following embodiments of the present application is a medium interface for interaction and information exchange between an application (APP) or an operating system (OS) and a user, which realizes conversion between internal forms of information and forms acceptable by the user. The user interface is source code written in a specific computer language such as Java or extensible markup language (XML), and the interface source code is parsed and rendered on an electronic device to finally present content recognizable by the user. A commonly used form of the user interface is a graphic user interface (GUI), which refers to a user interface displayed in a graphical manner and related to computer operation. It can be a visual interface element such as text, an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, and the like displayed in a display screen of the electronic device.

[0077] In some embodiments, the electronic device can generate a garment model according to garment data and generate a human body model according to human body data of the user. The garment model and the human body model can be 2D graphics. When performing virtual fitting, the electronic device can synthesize the garment model as a texture with the human body model, so as to render a fitting effect of the virtual image of the user wearing the garment. However, since the garment model is a static picture, the fitting effect rendered by the electronic device cannot present the deformation of the garment caused by the collision with the human body. That is to say, even if the virtual image of the user performs different fitting actions, the fitting effect rendered by the electronic device using the garment model is the same. The user cannot see the dynamic change of the garment when the virtual image performs different fitting actions.

[0078] The present application provides a virtual fitting method. The method can be applied to a communication system comprising a mobile terminal and a server. In response to an operation of the user selecting a garment and a fitting action to perform virtual fitting, the mobile terminal can send three dimensional (3D) image data of the user, 3D digital garment data, and action data to the server. The fitting action can be a single action, or can also be a sequence of actions comprising multiple actions. The server can calculate garment deformation information of the human body and the garment after collision under the corresponding fitting action by using a physical simulation technology, to obtain simulation data of the digital garment. The server can send the simulation data of the digital garment to the mobile terminal. Then, the mobile terminal can perform real-time rendering according to the simulation data, to display a fitting effect of the 3D image of the user wearing the digital garment to perform the fitting action. The fitting effect can realistically present garment information such as the style and fabric of the digital garment, and deformation information of details such as wrinkles caused by the collision between the digital garment and the 3D image.

[0079] The digital garment can represent a garment obtained by garment modeling a physical garment.

[0080] The method can realize providing a virtual fitting function on a mobile terminal through end-cloud cooperation, and present a high-precision dynamic virtual fitting effect. In this way, when a user shops in an online clothing market, the user can perform virtual fitting and view a 3D image matching the user's body type wearing a digital garment. The fitting effect can truly reflect the effect of the user wearing the corresponding garment. This can help the user understand the fitting effect of the garment before placing an order in the online clothing market, reduce the situation of buying an unsuitable garment in the online clothing market, and reduce the return rate of the clothing e-commerce.

[0081] The following introduces concepts related to embodiments of the present application.

[0082] Mesh

[0083] A mesh can refer to a mesh of a 3D model. The 3D model can be, for example, a human body model, a garment model, and the like. Mesh data can include data for representing a mesh of a 3D model. The mesh data can be used to render a mesh corresponding thereto. The 3D model can be composed of polygons. A complex polygon can be composed of multiple triangular faces. Therefore, the surface of a 3D model can be composed of multiple triangular faces connected to each other. In three-dimensional space, the set of points and edges of the triangular faces can be a mesh. For example, a human body mesh can include points and edges of triangular faces that compose the surface of a human body model. Human body mesh data can include coordinates of the points that compose the surface of the human body model, connection relationships between multiple points, and the like. A garment mesh can include points and edges of triangular faces that compose the surface of a garment model. Garment mesh data can include coordinates of the points that compose the surface of the garment model, connection relationships between multiple points, and the like.

[0084] The points of the triangular faces that compose the surface of a 3D model can be referred to as vertices. Each vertex can correspond to one or more pieces of information such as a normal, a UV texture coordinate, a color, and a tangent. These pieces of information can be used to render and draw the 3D model.

[0085] A 3D model can be obtained by modeling. Modeling can refer to establishing a mesh of a 3D model. The mesh can contain several main attributes (vertex coordinates, normals, texture coordinates, triangle drawing sequences, and the like). The process of establishing the mesh can be the process of drawing triangles.

[0086] The following introduces the architecture of a communication system and the hardware structure and software structure of a device related to the present application.

[0087] Figure 1 An exemplary architecture diagram of a communication system 10 is shown.

[0088] As shown in Figure 1 , the communication system 10 can include an electronic device 100 and a server 200. A communication connection can be established between the electronic device 100 and the server 200. The electronic device 100 can be a mobile terminal. For example, the electronic device 100 can be a mobile phone, a tablet computer, a smart watch, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. The specific type of the electronic device 100 is not limited in the embodiments of the present application.

[0089] The electronic device 100 and the server 200 can cooperate to provide a virtual fitting service for a user. The user can select a garment to be tried on the electronic device 100, and view a 3D image wearing the garment on the electronic device 100.

[0090] As shown in Figure 1 , the electronic device 100 can include an e-commerce application 210. The e-commerce application 210 can be an APP for providing an online garment purchase function. The name of the e-commerce application 210 is not limited in the embodiments of the present application. The e-commerce application 210 can include an account management module 211, a product display module 212, an order purchase module 213, a payment module 214, a 3D image management module 215, and a virtual fitting module 216.

[0091] The account management module 211 can be used to manage account information of a login account in the e-commerce application 210. The account information can include, but is not limited to, an account name, an account password, and the like. Based on the account management module 211, the e-commerce application 210 can provide functions such as account login, account logout, modification of the account name and the account password, and the like for the user.

[0092] The product display module 212 can be used to provide a product display function. The products can include garments (such as shirts, trousers, skirts, shoes, and the like). Based on the product display module 212, the e-commerce application 210 can display product information such as pictures, prices, parameters, and the like of the products. The product information displayed by the e-commerce application 210 can be uploaded by a merchant registered in the e-commerce application 210.

[0093] The order purchase module 213 can be used to provide a product ordering function. Based on the order purchase module 213, the user can place an order to purchase a product in the e-commerce application 210.

[0094] The payment module 214 can be configured to provide payment functions. Based on the payment module 214, a user can make a payment after placing an order in the e-commerce application 210.

[0095] The e-commerce application 210 provided by the embodiments of the present application is not limited in terms of the implementation of the account management function, the product display function, the order placement function, and the payment function.

[0096] The 3D avatar management module 215 can be configured to provide a 3D avatar creation function. Based on the 3D avatar management module 215, the e-commerce application 210 can create one or more 3D avatars. The one or more 3D avatars can include a 3D avatar of a user corresponding to a login account in the e-commerce application 210, and can also include 3D avatars of other users. For example, user 1 can log in to his own account in the e-commerce application 210 and create his own 3D avatar, so that he can use his own 3D avatar for virtual fitting in the future. User 1 can also create a 3D avatar of user 1's mother in the e-commerce application 210. In this way, user 1 can also use the 3D avatar of his mother for virtual fitting to purchase clothes for his mother.

[0097] A 3D avatar can be obtained by modeling a human body of a user. A 3D avatar of a user can realistically reflect the body shape of the user. That is, the body size information of a 3D avatar of a user is the same as or similar to the real body size information of the user at key body parts. The similarity of the body size information can mean that the difference between the body size information of the 3D avatar and the real body size information of the user is less than a preset threshold. The body size information can include one or more of the following: height, shoulder width, bust, waist, hip, arm circumference, arm length, neck circumference, abdominal circumference, hip circumference, thigh circumference, calf circumference, leg length, and the like. The embodiments of the present application are not limited in terms of the content of the body size information.

[0098] The method of modeling a human body to create a 3D avatar by the 3D avatar management module 215 will be described in detail in subsequent embodiments, which will not be expanded here.

[0099] In some embodiments, the 3D avatar management module 215 can be a module independent of the e-commerce application 210. For example, the 3D avatar management module 215 can be a system-level application provided by the electronic device 100. All e-commerce applications in the electronic device 100 can obtain a 3D avatar of a user through the 3D avatar management module 215, and use the 3D avatar to provide a virtual fitting function. In this way, a user can create a 3D avatar only once in the electronic device 100, without the need to create a 3D avatar multiple times in different e-commerce applications. This can simplify the operation of creating a 3D avatar for a user and improve the user experience.

[0100] The virtual fitting module 216 can be configured to provide a function of showing a virtual fitting effect. The virtual fitting module 216 can include a 3D rendering module 216A, a digital garment simulation software development kit (SDK) 216B, a motion management module 216C, and a scene management module 216D.

[0101] The motion management module 216C can be configured to manage motions required for virtual fitting. In some embodiments, the motions can be pre-stored single motions or motion sequences. In other embodiments, the motions can also be motion sequences collected in real time through motion capture technology. The motion sequence is a set of motions with a sequence.

[0102] Based on the motion management module 216C, the e-commerce application 210 can provide one or more motion options for a user to select a fitting motion when performing virtual fitting. In the scene of virtual fitting, the 3D avatar can move according to the fitting motion selected by the user, thereby presenting a fitting effect of the 3D avatar wearing a garment and performing a fitting motion.

[0103] Based on the motion management module 216C, the e-commerce application 210 can also provide a function of recording a fitting motion. For example, the e-commerce application 210 can call a camera in the electronic device 100 to collect images containing a user motion. Then, the e-commerce application 210 can analyze the user motion from the images through the motion management module 216C. The user motion can be used as a fitting motion when performing virtual fitting.

[0103] The scene management module 216D can be configured to manage scenes required for virtual fitting. The scene can represent a background when the 3D avatar performs virtual fitting. For example, the scene can include a T-stage, a stage, a square, a lawn, a beach, a house, and the like. The type of the scene is not limited in the embodiments of the present application.

[0104] The digital garment simulation SDK 216B can be configured to interact with the server 200. The digital garment simulation SDK 216B can be referred to as a processing module of the electronic device 100.

[0105] In some embodiments, the digital garment simulation SDK 216B can upload 3D avatar data, 3D digital garment data, and motion data to the server 200.

[0106] The 3D avatar data can include mesh data of a human body. In some embodiments, the 3D avatar data can also include skeleton data. The human body model constructed by the 3D avatar data can represent the 3D avatar corresponding to the 3D avatar data.

[0107] The 3D digital clothing data can include clothing mesh data and clothing parameter information. The clothing parameter information can include, but is not limited to, a stretch coefficient, a bending coefficient, a friction coefficient, and the like. The clothing parameter information can be determined according to the physical material of the clothing fabric. The 3D digital clothing data can be obtained by clothing modeling of the clothing.

[0108] The action data can include data corresponding to the fitting action. For example, the action data can be complete skeleton data of the fitting action. Alternatively, the action data can be an action ID corresponding to the fitting action.

[0109] The digital clothing simulation SDK 216B can obtain the action data from the action management module 216C. Then, the digital clothing simulation SDK 216B can send the action data to the server 200.

[0110] In some embodiments, the digital clothing simulation SDK 216B can obtain simulation data of the clothing obtained by the physical simulation technology from the server 200. The simulation data of the clothing can be used to represent the shape of the clothing and the human body after collision under the corresponding fitting action.

[0111] In a possible implementation, the server 200 can include a 3D digital clothing simulation module 230. The digital clothing simulation SDK 216B can specifically interact with the 3D digital clothing simulation module 230.

[0112] In some embodiments, the digital clothing simulation SDK 216B can also be used to provide a coding and decoding function to code and decode data exchanged between the digital clothing simulation SDK 216B and the server 200. For example, the digital clothing simulation SDK 216B can code the data and send the coded data to the 3D digital clothing simulation module 230. Then, the 3D digital clothing simulation module 230 can decode the received data. Coding the data by the digital clothing simulation SDK 216B can reduce the amount of uplink data. The digital clothing simulation SDK 216B can receive data sent from the 3D digital clothing simulation module 230. The data can be coded by the 3D digital clothing simulation module 230. The digital clothing simulation SDK 216B can decode the received data. The coding and decoding function can reduce the transmission bandwidth between the digital clothing simulation SDK 216B and the server 200.

[0113] In some embodiments, the digital clothing simulation SDK 216B can also be used to pre-process the simulation data of the clothing before rendering. For example, the digital clothing simulation SDK 216B can perform normal calculation on the simulation data of the clothing.

[0114] The 3D rendering module 216A can be used to perform 3D rendering based on motion data, scene data, 3D image data, and clothing simulation data to display a virtual try-on effect. In one possible implementation, the image rendered by the 3D rendering module 216A can be a view from a fixed virtual camera perspective, or a view after switching the virtual camera perspective according to user control. The rendered image can also be zoomed in or out. Optionally, the rendered image can also be interactive based on user input. For example, the user input could be to apply a directional force to the clothing. The 3D rendering module 216A can then simulate a wind effect during 3D rendering based on this user input, presenting the effect of wind blowing the clothing in a specified direction.

[0115] Among them, the digital clothing simulation SDK 216B can send simulation data of clothing obtained from the server 200 to the 3D rendering module 216A. The motion management module 216C can send motion data to the 3D rendering module 216A. The scene management module 216D can send scene data to the 3D rendering module 216A.

[0116] In some embodiments, the electronic device 100 may further include a storage module ( Figure 1 (Not shown in the image). The electronic device 100 can store simulation data of clothing in the combination of 3D image + clothing + try-on action in a storage module. This simulation data can be sent by the server 200. Thus, when it is detected that the user selects the same 3D image + clothing + try-on action again for virtual try-on, the e-commerce application 210 can retrieve the clothing simulation data from the aforementioned storage module without requesting the server 200 to perform fabric simulation again. The above embodiment can save computing resources of the server 200 and bandwidth of the electronic device 100.

[0117] In some embodiments, the providers of the digital clothing simulation SDK 216B and the e-commerce application 210 may be the same or different. The providers of the 3D image management module 215, the 3D rendering module 216A, the motion management module 216C, and the scene management module 216D may be the same or different.

[0118] Server 200 may include a 3D digital clothing management module 220 and a 3D digital clothing simulation module 230.

[0119] The 3D digital garment simulation module 230 can be configured to provide high-precision cloth simulation services. The cloth simulation can also be referred to as garment simulation. The cloth simulation can be a process of calculating garment mesh deformation by calculating the collision between the human body mesh and the garment mesh. The 3D digital garment simulation module 230 can determine the simulation data of the garment according to the 3D image data, the 3D digital garment data, and the action data. The simulation data of the garment can include garment mesh data. The garment mesh data in the simulation data of the garment can be the mesh data after the garment is deformed due to collision with the human body, and can reflect the shape of the garment and the human body after collision under the corresponding fitting action.

[0120] The 3D digital garment management module 220 can be configured to manage the 3D digital garment data. The 3D digital garment management module 220 can store the 3D digital garment data (such as garment mesh data and garment parameter information) obtained through garment modeling.

[0121] In some embodiments, the e-commerce application 210 can communicate with the 3D digital garment management module 220 to determine whether a garment has been modeled and has corresponding 3D digital garment data. When it is determined that a garment has 3D digital garment data, the e-commerce application 210 can provide an operation control for virtual fitting when displaying the garment. In this way, the user can use the operation control to virtually try on the garment. When it is determined that a garment does not have 3D digital garment data, the e-commerce application 210 can not provide an operation control for virtual fitting when displaying the garment. That is, the user cannot virtually try on the garment.

[0122] In some embodiments, during the virtual fitting process, the digital garment simulation SDK 216B in the e-commerce application 210 can send the identification information (such as the garment ID) of the garment to the 3D digital garment simulation module 230. Then, the 3D digital garment simulation module 230 can obtain the 3D digital garment data of the garment from the 3D digital garment management module 220 according to the identification information of the garment, and perform cloth simulation on the garment. The data amount of the identification information of the garment is much smaller than the data amount of the garment mesh data and the garment parameter information. It can be seen that the above method of transmitting the identification information of the garment during the virtual fitting process can reduce the data transmission amount between the terminal and the cloud, and save the transmission bandwidth between the electronic device 100 and the server 200.

[0123] In some embodiments, the 3D digital clothes simulation module 230 can send the simulation data of the clothes after cloth simulation to the 3D digital clothes management module 220. The 3D digital clothes management module 220 can store the simulation data of the clothes in the case of 3D figure + clothes + fitting action combination. When the same fitting action is needed to be performed on the same or similar 3D figure to put on the same clothes for cloth simulation again, the 3D digital clothes simulation module 230 can obtain the simulation data of the clothes from the 3D digital clothes management module 220, without the need to perform the specific calculation process of cloth simulation again. The similar 3D figure can mean that the difference between the body size information of the 3D figure is less than a preset threshold. It can be seen that the above-mentioned embodiments can reduce the repeated cloth simulation process of the server 200, and save the computing resources of the server 200.

[0124] In some embodiments, the electronic device 100 can contain a 3D digital clothes simulation module. In this way, the electronic device 100 can complete cloth simulation and rendering of virtual fitting effect without interacting with the server 200.

[0125] In some embodiments, the server 200 can contain a 3D rendering module, a scene management module. In this way, the server 200 can complete cloth simulation and rendering of virtual fitting effect. The server 200 can send the rendering result to the electronic device 100. The electronic device 100 can display the virtual fitting effect according to the received rendering result.

[0126] The electronic device 100 and the server 200 described above can also contain more or less modules. The present application embodiments do not limit this.

[0127] Here, the software and hardware structure of the electronic device 100 is introduced.

[0128] Figure 2A An exemplary hardware structure schematic diagram of the electronic device 100 is shown.

[0129] As Figure 2AAs shown, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0130] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0131] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors. In some embodiments, the processor 110 can be a system on chip (SOC).

[0132] Among them, the controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, complete the control of fetching instructions and executing instructions.

[0133] The processor 110 can also include a memory for storing instructions and data. In some examples, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has just used or is using repeatedly. If the processor 110 needs to use the instructions or data again, it can call them directly from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thus improving the efficiency of the system.

[0134] The USB interface 130 is an interface that meets the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect a headset to play audio through the headset.

[0135] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. The charging management module 140 can charge the battery 142 while also providing power to the electronic device through the power management module 141.

[0136] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160, etc.

[0137] The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0138] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0139] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive an electromagnetic wave by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic wave, and transfer the processed electromagnetic wave to the modem processor to be demodulated. The mobile communication module 150 can also amplify a signal modulated by the modem processor, and radiate the amplified signal as an electromagnetic wave through the antenna 1.

[0140] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrated with at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave through the antenna 2, performs frequency modulation and filtering on the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and radiate the processed signal as an electromagnetic wave through the antenna 2.

[0141] The electronic device 100 can implement a display function through a GPU, a display screen 194, an application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.

[0142] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 can include one or N display screens 194, N being a positive integer greater than 1.

[0143] The electronic device 100 can implement a photographing function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, etc.

[0144] ISP is used to process the data feedback by camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye.

[0145] Camera 193 is used to capture still images or videos. In some embodiments, electronic device 100 can include 1 or N cameras 193, N being a positive integer greater than 1.

[0146] In some embodiments, the e-commerce application in electronic device 100 can call camera 193 to take an image containing user actions. Then, electronic device 100 can use motion capture technology to identify user actions from the above-mentioned image, and determine the user actions as fitting actions in the virtual fitting process.

[0147] In some embodiments, electronic device 100 can also perform 3D rendering of the virtual fitting effect through GPU.

[0148] Digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0149] NPU is a neural network (NN) calculation processor, which can quickly process input information by borrowing the structure of biological neural network, such as borrowing the transmission mode between human brain neurons, and can also constantly self-learn. Through NPU, electronic device 100 can realize intelligent cognition and other applications, such as image recognition, face recognition, speech recognition, text understanding, etc.

[0150] In some embodiments, electronic device 100 can model the human body through NPU to obtain the 3D image of the user. For example, NPU can receive body size information of the user at key body parts, and generate a 3D image of the user according to the above-mentioned body size information. For another example, NPU can receive one or more images containing a human body, and generate a 3D image of the user according to the above-mentioned images. The embodiments of the present application do not limit the method of modeling the human body by the above-mentioned NPU.

[0151] External memory interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of electronic device 100. The external storage card communicates with processor 110 through external memory interface 120 to realize data storage function. For example, save music, video and other files in the external storage card.

[0152] The internal memory 121 can be used to store computer-executable program codes including instructions. The processor 110 performs various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program (e.g., a sound play function, an image play function, etc.) required for at least one function, etc. The data storage area can store data (e.g., audio data, a phonebook, etc.) created during the use of the electronic device 100, etc. In addition, the internal memory 121 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one of a magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0153] In some embodiments, the internal memory 121 can further be used to store one or more of the following data: 3D avatar data, simulation data of a garment, motion data of a fitting motion, scene data of a fitting scene, identification information of a garment, etc.

[0154] The electronic device 100 can implement an audio function through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, etc. For example, music play, recording, etc.

[0155] The audio module 170 is used to convert digital audio information into an analog audio signal output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used to encode and decode an audio signal. In some examples, the audio module 170 can be disposed in the processor 110, or part of the functions of the audio module 170 can be disposed in the processor 110. The speaker 170A, also known as a "loudspeaker", is used to convert an audio electrical signal into a sound signal. The receiver 170B, also known as a "earpiece", is used to convert an audio electrical signal into a sound signal. The microphone 170C, also known as a "microphone", "sound transducer", is used to convert a sound signal into an electrical signal. The earphone interface 170D is used to connect a wired earphone.

[0156] The sensor module 180 can include a pressure sensor, a gyro sensor, a barometric sensor, a magnetic sensor, an acceleration sensor, a gravity sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0157] The gyroscope sensor can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor. The electronic device 100 can determine the deflection angle of the electronic device 100 by the gyroscope sensor.

[0158] The acceleration sensor can detect the magnitude of acceleration of the electronic device 100 in each direction (generally three axes). In some embodiments, the acceleration sensor can be used to identify the posture of the electronic device 100, which can be applied to landscape / portrait switching, pedometer, and other applications.

[0159] The gravity sensor can be used to determine the tilt angle of the electronic device 100 relative to the horizontal plane. In some embodiments, the screen state of the electronic device 100 can be determined by the gravity sensor, so as to adjust the screen to keep it horizontal.

[0160] In some embodiments, the electronic device 100 can determine the moving distance of the electronic device 100 in a period of time by the acceleration sensor and the gravity sensor.

[0161] The keys 190 include a power key, a volume key, and the like. The motor 191 can generate a vibration prompt. The indicator 192 can be an indicator light, which can be used to indicate the charging state, the power change, and can also be used to indicate messages, missed calls, notifications, and the like.

[0162] The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into or pulled out of the SIM card interface 195 to realize contact and separation with the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, and N is a positive integer greater than 1. The electronic device 100 interacts with the network through the SIM card to realize functions such as call and data communication. In some examples, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0163] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0164] Figure 2B is a software structure block diagram of the electronic device 100 of the embodiment of the present application.

[0165] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the software architecture of the electronic device 100 is divided into four layers: the application layer, the framework layer, the core service layer, and the system service layer. ​The system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android Runtime and system libraries, and the kernel layer.

[0166] The application layer can include a series of application packages.

[0167] As shown in Figure 2B , the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, short message, e-commerce application, etc. Among them, the e-commerce application can provide online clothing purchase function, etc.

[0168] The application framework layer provides APIs and programming framework for the application layer. The application framework layer includes some pre-defined functions.

[0169] As shown in Figure 2B , the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, activity manager, etc.

[0170] The window manager is used to manage window programs. The window manager can get the size of the display screen, judge whether there is a status bar, lock the screen, intercept the screen, etc.

[0171] The content provider is used to store and obtain data, and make the data accessible to the application. The data can include video, image, audio, dialed and received calls, browsing history and bookmarks, phonebook, etc.

[0172] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build an application. The display interface can be composed of one or more views. For example, the display interface including the short message notification icon can include a view for displaying text and a view for displaying pictures.

[0173] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of call state (including call connection, call hang-up, etc.).

[0174] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc.

[0175] The notification manager enables an application to display notification information in the status bar (a pull-down notification bar), which can be used to convey a message of the notification type, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the completion of a download, a message reminder, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification in the form of a dialog window appearing on the screen. For example, a text message is prompted in the status bar, a prompt sound is emitted, the electronic device vibrates, the indicator light flashes, etc.

[0176] The activity manager is used to manage activities (activity) and is responsible for the startup, switching, scheduling of components in the system, and the management and scheduling of applications. The activity manager can be called by the upper layer application to open the corresponding activity.

[0177] The Android Runtime includes a core library and a virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

[0178] The core library includes two parts: one is the function function called by the java language, and the other is the core library of Android.

[0179] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java file of the application layer and the application framework layer into a binary file. The virtual machine is used to manage the object lifecycle, stack management, thread management, security and exception management, and garbage collection.

[0180] The system library can include multiple functional modules. For example: surface manager, media library, three-dimensional graphics processing library (for example: OpenGL ES), two-dimensional graphics engine (for example: SGL) and the like.

[0181] The surface manager is used to manage the display subsystem and provides 2D and 3D layer fusion for multiple applications.

[0182] The media library supports multiple commonly used audio, video format playback and recording, and static image files, etc. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0183] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing. Among them, the electronic device 100 can realize 3D rendering of virtual fitting based on the three-dimensional graphics processing library, and present the fitting effect of the user's 3D image wearing clothes and performing fitting actions on the display screen.

[0184] The 2D graphics engine is a drawing engine for 2D drawing.

[0185] The kernel layer is a layer between hardware and software. The kernel layer at least includes display drivers, camera drivers, audio drivers, and sensor drivers.

[0186] Figure 3 An exemplary structure diagram of a server 200 provided by an embodiment of the application is shown.

[0187] As shown in Figure 3 , the server 200 can include one or more processors 310, a memory 311, a communication interface 312, a transmitter 314, a receiver 315, a coupler 316, and an antenna 317. These components can be connected through a bus 313 or other means. Figure 3 Taking the connection through the bus as an example. Among them:

[0188] The communication interface 312 can be used for the server 200 to communicate with other electronic devices, such as Figure 1 the electronic device 100 shown in the figure, and the like. Specifically, the communication interface 312 can be a 3G communication interface, a long-term evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, and the like. Not limited to wireless communication interfaces, the server 200 can also be configured with wired communication interfaces 312 to support wired communication.

[0189] In some embodiments of the application, the transmitter 314 and the receiver 315 can be regarded as a wireless modem. The transmitter 314 can be used for transmitting processing of signals output by the processor 310. The receiver 315 can be used to receive signals. In the server 200, the number of transmitters 314 and receivers 315 can be one or more. The antenna 317 can be used to convert electromagnetic energy in the transmission line into electromagnetic waves in free space, or convert electromagnetic waves in free space into electromagnetic energy in the transmission line. The coupler 316 can be used to divide the mobile communication signal into multiple paths and distribute it to multiple receivers 315. It can be understood that the antenna 317 of the server 200 can be implemented as a large-scale antenna array.

[0190] The memory 311 is coupled to the processor 310 and stores various software programs and / or sets of instructions. Specifically, the memory 311 can include a high-speed random access memory and can also include a non-volatile memory such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

[0191] The memory 311 can store an operating system (hereinafter referred to as system) such as an embedded operating system such as VxWorks, QNX, or the like. The memory 311 can also store a network communication program that can be used to communicate with one or more electronic devices such as the electronic device 100.

[0192] In embodiments of the present application, the memory 311 can be used to store clothing data, simulation data of clothing, and action data of a fitting action. The clothing data can include one or more of the following: identification information of the clothing (such as clothing ID, etc.), 3D digital clothing data. The 3D digital clothing data can include clothing mesh data and clothing parameter information. The action data can include one or more of the following: identification information of the action (such as action ID, etc.), bone data of the action.

[0193] Optionally, the memory 311 can also be used to store base model data of a human body base model consistent with the side of the electronic device 100, so that the server 200 can generate 3D image data of a user consistent with the side of the electronic device 100 according to the base model data.

[0194] In embodiments of the present application, the processor 310 can be used to read and execute computer-readable instructions. Specifically, the processor 310 can be used to call a program stored in the memory 311 and execute instructions contained in the program. The program can be, for example, an implementation program of cloth simulation provided by one or more embodiments of the present application.

[0195] It should be noted that Figure 3 The server 200 shown is only one implementation of an embodiment of the present application, and in actual applications, the server 200 can include more or fewer components, which are not limited here.

[0196] Based on Figure 1 The communication system 10 shown, some virtual fitting scenarios provided by embodiments of the present application are introduced below.

[0197] Figures 4A-4E Some 3D image creation scene schematic diagrams are exemplarily shown.

[0198] As Figure 4AAs shown, the electronic device 100 can display a user interface 410. The user interface 410 can include application icons. For example, an e-commerce application icon 411. In response to an operation on the e-commerce application icon 411, the electronic device 100 can run the e-commerce application, display Figure 4B As shown, the user interface 420 can be a product display interface in the e-commerce application. The user interface 420 can include a product option 421, a product option 422, a product option 423, a product option 424, and so on.

[0199] As shown, the user interface 420 can be a product display interface in the e-commerce application. The user interface 420 can include a product option 421, a product option 422, a product option 423, a product option 424, and so on. Figure 4B As shown, the user interface 420 can be a product display interface in the e-commerce application. The user interface 420 can include a product option 421, a product option 422, a product option 423, a product option 424, and so on.

[0200] The product option 422 can be a product option corresponding to a shirt. The product option 422 can include a virtual fitting control 422A. The virtual fitting control 422A can be used to virtually try on the clothing corresponding to the product option 422.

[0201] The product option 423 can be a product option corresponding to a skirt. The product option 423 can also include a virtual fitting control.

[0202] The product option 424 can be a product option corresponding to a short-sleeved shirt. The product option 424 does not include a virtual fitting control. This indicates that the clothing corresponding to the product option 424 cannot be virtually tried on.

[0203] In one possible implementation, the e-commerce application in the electronic device 100 can decide whether to provide the virtual fitting control in the product option corresponding to a piece of clothing according to whether the clothing can be virtually tried on. The e-commerce application can determine whether the clothing can be virtually tried on according to whether the 3D digital clothing data of the clothing is stored in the server 200. If the 3D digital clothing data of the clothing is stored in the server 200, the clothing can be virtually tried on. Otherwise, the clothing cannot be virtually tried on.

[0204] The user interface 420 can also include a my control 425. In response to an operation on the my control 425, the electronic device 100 can display Figure 4C As shown, the user interface 430 can include a user information display area 431, a 3D avatar control 432, and so on.

[0205] As shown, the user interface 430 can include a user information display area 431, a 3D avatar control 432, and so on. Figure 4C As shown, the user interface 430 can include a user information display area 431, a 3D avatar control 432, and so on.

[0206] The user information display area 431 can be used to display account information of a logged-in account in the e-commerce application. For example, an account avatar, an account name, and the like.

[0207] The 3D avatar control 432 can be used to trigger the electronic device 100 to open a management interface of 3D avatars. In response to an operation on the 3D avatar control 432, the electronic device 100 can display Figure 4D the user interface 440 as shown.

[0208] As Figure 4D shown, the user interface 440 can include a new control 441, and an existing 3D avatar area 442.

[0209] The new control 441 can be used to create a new 3D avatar.

[0210] The existing 3D avatar area 442 can display options corresponding to 3D avatars that have been created by the user (e.g., user 1) in the e-commerce application. For example, a 3D avatar option 442A and a 3D avatar option 442B. The 3D avatar option 442A can correspond to a 3D avatar of the user 1 himself. The 3D avatar option 442B can correspond to a 3D avatar of the mother of the user 1.

[0211] In response to Figure 4D an operation on the new control 441, the electronic device 100 can display Figure 4E the user interface 450 as shown.

[0212] As Figure 4E shown, the user interface 450 can include an information input area 451, a confirm control 452, and a cancel control 453.

[0213] The information input area 451 can be used to input information for creating a 3D avatar. The information for creating a 3D avatar described above can include one or more body size information. For example, height, weight, shoulder width, bust, waist, hip, arm circumference, arm length, and the like. Embodiments of the present application do not limit the information for creating a 3D avatar described above.

[0214] The confirm control 452 can be used to trigger the electronic device 100 to create a 3D avatar according to the information input in the information input area 451.

[0215] The cancel control 453 can be used to cancel the creation of a 3D avatar.

[0216] Not limited to using Figure 4E one or more body size information as shown to create a 3D avatar, the electronic device 100 can also use other methods for human modeling to create a 3D avatar. For example, the electronic device 100 can also use one or more images containing a human body to create a 3D avatar. Then, in response to an operation onFigure 4D In response to the operation of the new control 441 as shown, the electronic device 100 can provide an image upload portal to prompt the user to upload one or more images containing a human body. Then, the electronic device 100 can perform human modeling according to the one or more images containing a human body uploaded by the user to create a 3D avatar.

[0217] The electronic device 100 can determine and save 3D avatar data after creating the 3D avatar. The 3D avatar data can include human mesh data and skeleton data. The 3D avatar data can be used for virtual fitting, so that the user can view the fitting effect of the 3D avatar wearing a garment.

[0218] The portal for creating a 3D avatar can be provided by a system-level application in the electronic device 100 in addition to being provided by an e-commerce application. That is, the user can create a 3D avatar in each different e-commerce application, and can also create a 3D avatar in a system-level application that provides a portal for creating a 3D avatar. The system-level application can provide an interface for obtaining a 3D avatar. Each e-commerce application in the electronic device 100 can obtain a 3D avatar from the system-level application through the interface for obtaining a 3D avatar.

[0219] As described above Figures 4A-4E As shown in the scenario, the electronic device 100 can provide a function of creating a 3D avatar to facilitate the user to create a 3D avatar that matches the body shape of the user or another user. In this way, in the process of virtual fitting, the user can view the fitting effect of the 3D avatar wearing a garment. The fitting effect can truly reflect the effect of the user corresponding to the 3D avatar wearing the garment, and can more accurately help the user to understand whether the garment is suitable for the user or another user.

[0220] Figures 5A-5H Some virtual fitting scenarios are exemplarily shown.

[0221] As Figure 5A shown, the electronic device 100 can display a user interface 420. The user interface 420 can include a product option 422. The product option 422 can include a virtual fitting control 422A. Other controls in the user interface 420 can refer to the descriptions of the preceding embodiments.

[0222] In response to Figure 5A the operation of the product option 422, the electronic device 100 can display a user interface 510 as shown. The user interface 510 can be a display interface of a garment corresponding to the product option 422. Figure 5B As

[0223] shown, the electronic device 100 can display a user interface 520. The user interface 520 can include a product option 522. The product option 522 can include a virtual fitting control 522A. Other controls in the user interface 520 can refer to the descriptions of the preceding embodiments. Figure 5BAs shown, the user interface 510 may include an image display area 511, a virtual try-on control 512, a shopping cart control 513, and a purchase control 514, etc.

[0224] Image display area 511 can be used to display images of clothing.

[0225] The virtual fitting control 512 can be used to display clothing (i.e., clothing) on ​​the user interface 510. Figure 5A Virtual try-on is available for the clothing item corresponding to product option 422 shown.

[0226] The shopping cart control 513 can be used to add clothing displayed on the user interface 510 to the shopping cart.

[0227] Purchase control 514 can be used to purchase clothing displayed in user interface 510.

[0228] In some embodiments, the electronic device 100 stores multiple 3D images. For example, as described above. Figure 4D The 3D images corresponding to 3D image options 442A and 442B are shown. In response to... Figure 5B The virtual try-on control shown is 512 or Figure 5A The operation of the virtual fitting control 422A shown can be displayed on the electronic device 100. Figure 5C The user interface 520 is shown. User interface 520 may include multiple options corresponding to 3D avatars in electronic device 100, allowing the user to select the 3D avatar for virtual try-on. For example, user interface 520 may include 3D avatar option 521 and 3D avatar option 522. 3D avatar option 521 may be the option corresponding to the user's own 3D avatar. 3D avatar option 522 may be the option corresponding to the user's mother's 3D avatar. Understandably, if a user needs to buy clothing for themselves, they can select their own 3D avatar for virtual try-on. If a user needs to buy clothing for their mother, they can select their mother's 3D avatar for virtual try-on.

[0229] The user interface 520 may also include a confirm control 523 and a cancel control 524. The confirm control 523 can be used to trigger the electronic device 100 to perform a virtual try-on based on the 3D image corresponding to the selected 3D image option in the user interface 520. The cancel control 524 can be used to cancel the virtual try-on.

[0230] In some embodiments, the electronic device 100 stores only one 3D image. In response to... Figure 5B The virtual try-on control shown is 512 or Figure 5A The virtual try-on control 422A shown allows the electronic device 100 to directly use the 3D image stored in the electronic device 100 for virtual try-on without displaying the actual device. Figure 5CThe user interface shown is simply for users to select a 3D avatar.

[0231] In some embodiments, the electronic device 100 does not store a 3D avatar. That is, the user has not yet created a 3D avatar. In response to Figure 5B The virtual try-on control shown is 512 or Figure 5A The operation of the virtual try-on control 422A shown can prompt the user to create a 3D avatar. For example, the electronic device 100 can display the aforementioned... Figure 4E The user interface 450 shown in the image instructs the user to create a 3D avatar. After the user creates the 3D avatar, the electronic device 100 can use the user-created 3D avatar for virtual try-on.

[0232] For example, in Figure 5C When the 3D image option 521 is selected, the electronic device 100 can display [the desired image] in response to the operation of the confirmation control 523. Figure 5D The user interface shown is 530.

[0233] like Figure 5D As shown, the user interface 530 may include a fitting effect display area 531, a pause control 532, a playback progress bar 533, a recommended clothing option area 534, an action option area 535, and a scene option area 536.

[0234] The fitting room effect display area 531 can be used to display the fitting room scene and the fitting room effect of a 3D avatar wearing the clothing and performing the fitting room action. The aforementioned fitting room effect can be obtained by the electronic device 100 through 3D rendering based on the 3D avatar data, action data, and clothing simulation data. The aforementioned clothing simulation data can be obtained by the electronic device 100 from the server 200.

[0235] The pause control 532 can be used to pause the playback of the try-on effect.

[0236] The playback progress bar 533 can be used to indicate the current playback progress of the virtual try-on effect.

[0237] It should be noted that, in some embodiments, the above-mentioned fitting effect can be presented in the form of video playback. The fitting action can be a sequence of actions with a sequential order. During the video playback presenting the fitting effect, the 3D avatar can wear the clothing being tried on and perform the actions in the sequence in sequence.

[0238] In this way, users can view the dynamic virtual try-on effect. Furthermore, users can use the pause control 532 to pause the playback of the try-on effect on the electronic device 100. Users can also view and adjust the playback progress of the try-on effect using the playback progress bar 533.

[0239] In some embodiments, the fitting effect displayed in the fitting effect display region 531 can also be static. For example, the fitting action can be a single action. The electronic device 100 can display, in the fitting effect display region 531, a 3D rendering image of the 3D figure wearing the clothes tried on by the user to perform a single fitting action.

[0240] The recommended clothes option region 534 can include one or more clothes options. The electronic device 100 can perform virtual fitting of the clothes corresponding to the clothes option selected in the recommended clothes option region 534 with the clothes combination displayed in the aforementioned Figure 5B user interface 510. For example, the recommended clothes option region 534 can include a shirt option, a skirt option, and a clothes 1 option. The clothes corresponding to the shirt option can be the shirt displayed in the aforementioned Figure 5B user interface 510. Among them, the shirt option and the skirt option are in the selected state. The electronic device 100 can perform virtual fitting of the shirt corresponding to the shirt option and the skirt corresponding to the skirt option. As can be seen in the fitting effect display region 531 shown in Figure 5D , the 3D figure wears the shirt and the skirt selected by the user for virtual fitting, and displays the upper body effect of the shirt and the skirt to the user.

[0241] In some embodiments, in the case of virtual fitting of a plurality of clothes combinations, the electronic device 100 can determine the hierarchical information of the plurality of clothes. The aforementioned hierarchical information can represent the wearing order of the plurality of clothes on the 3D figure. For example, the shirt can be worn on the skirt, partially covering the skirt. Or the skirt can be worn on the shirt, partially covering the shirt. Among them, the wearing order of the plurality of clothes can be specified by the user. The electronic device 100 can provide a wearing order specifying control in the user interface 530, so that the user can specify the wearing order of the plurality of clothes through the wearing order specifying control. The embodiments of the present application do not limit the user operation for specifying the wearing order.

[0242] Optionally, in the case of virtual fitting of a plurality of clothes combinations, the electronic device 100 can also determine the wearing information such as the position information and the fitting state of one or more clothes in the plurality of clothes when being tried on. For example, when the clothes tried on include a skirt, the electronic device 100 can determine the height of the skirt worn on the 3D figure (such as being worn at a high waist position, or a middle waist position, or a low waist position, etc.). For another example, when the clothes tried on include a coat, the electronic device 100 can determine the fitting state of the coat (such as the buttons of the coat being fastened, or the buttons of the coat being unfastened). The embodiments of the present application do not limit the specific content of the wearing information. Among them, the position information and the fitting state of the wearing information can be specified by the user. The embodiments of the present application do not limit the user operation for specifying the wearing information.

[0243] Understandably, users are not limited to selecting the above-mentioned skirt option in the recommended clothing option area 534. They can also select more clothing options in the recommended clothing option area 534 to match the shirts displayed in the user interface 510 for virtual try-on.

[0244] In other words, users can choose to virtually try on a single piece of clothing, or they can choose to mix and match multiple pieces of clothing to see how the outfits look.

[0245] Optionally, the electronic device 100 can also select the clothing corresponding to the selected clothing option in the recommended clothing option area 534. Figure 5B The user interface 510 displays clothing replacements for virtual try-on. For example, if the selected clothing option in the recommended clothing option area 534 includes a top, this top can replace the shirt displayed in the user interface 510. The try-on effect display area 531 can show the 3D avatar wearing the replacement shirt. In some embodiments, the electronic device 100 can provide replacement controls in the user interface 530. These replacement controls can be used to replace the clothing currently being tried on by the 3D avatar with one or more garments in a virtual try-on scenario. This application embodiment does not limit the display style of the replacement controls.

[0246] In other words, during the virtual try-on process, users can choose one or more garments to replace the ones already worn by the 3D avatar. For example, users can adjust the size and color of the garment worn by the 3D avatar, or change the garment to a different style.

[0247] In some embodiments, the electronic device 100 can be configured to allow the user to initially select clothing for trying on (as described above). Figure 5B The user interface 510 shows a shirt to determine recommended clothing, and displays clothing options corresponding to the recommended clothing in the recommended clothing option area 534. For example, the recommended clothing may include clothing that can be matched with the clothing initially selected by the user for trying on. This application embodiment does not limit the implementation method of the electronic device 100 determining the recommended clothing.

[0248] The action option area 535 may contain one or more action options, which can be used by the user to select the desired try-on action. For example, the action option area 535 may contain action 1, action 2, and action 3 options. The try-on action corresponding to an action option can be a single action or a sequence of multiple actions in a specific order. These try-on actions can be preset. Optionally, these try-on actions can be captured in real time using motion capture technology.

[0249] In a possible implementation, the electronic device 100 can provide a try-on action recording function for the user to record a try-on action. In this case, the electronic device 100 invokes the camera to collect the user action by using the motion capture technology. The electronic device 100 can store the motion data of the collected user action, and provide a corresponding action option in the action option area 535. In this way, the user can select the action option corresponding to the try-on action recorded by the user in the action option area 535, so as to view the try-on effect of the 3D image wearing the clothes to be tried on and performing the try-on action recorded by the user.

[0250] That is, the user can record a try-on action that the user needs or likes, and select to use the recorded try-on action in the virtual try-on.

[0251] In another possible implementation, the electronic device 100 can provide a try-on action real-time matching function. The try-on action real-time matching function can be used to match the action performed by the 3D image with the user action in real time in the virtual try-on. That is, the 3D image can imitate the action of the user in real time while wearing the clothes to be tried on. For example, the user walks forward, and the 3D image also walks forward. The user turns around, and the 3D image also turns around. The user raises the arm, and the 3D image also raises the arm. In the process of virtual try-on, the electronic device 100 can keep the camera open, collect the user action in real time by using the motion capture technology, and determine the user action as the try-on action. The electronic device 100 can render the try-on effect of the 3D image wearing the clothes and performing the user action in real time by using 3D rendering.

[0252] That is, in the process of virtual try-on, the user can perform various actions to view the try-on effect of the 3D image wearing the clothes and performing the corresponding action.

[0253] The scene option area 536 can include one or more scene options, which can be used for the user to select a scene for virtual try-on. For example, the scene option area 536 can include a scene 1 option, a scene 2 option, and a scene 3 option. The embodiments of the present application do not limit the above scenes.

[0254] Here, the dynamic try-on effect presented in the try-on effect display area 531 is taken as an example for description.

[0255] Comparison Figure 5D and Figure 5E It can be seen that Figure 5E the try-on action performed by the 3D image in the try-on effect display area 531 shown in FIG. 6B is different from Figure 5D the try-on action performed by the 3D image in the try-on effect display area 531 shown in FIG. 6A. Figure 5E The play progress bar 533 shown in FIG. 6B is longer than Figure 5D the play progress bar 533 shown in FIG. 6A. That is Figure 5EThe playing progress of the try-on effect shown in FIG. 13B is more than that of the try-on effect shown in FIG. 13A. Figure 5D It can be seen that, in the case of different try-on actions, the 3D image and the clothes will produce different collisions, and thus the clothes will produce different deformations. The electronic device 100 can perform 3D rendering according to the simulation data of the clothes in different try-on actions, to present the try-on effect of the 3D image wearing the clothes in different try-on actions. The user can view the deformation of the clothes when the 3D image performs different try-on actions. For example, the try-on actions that the 3D image needs to perform can include a first action and a second action. The electronic device 100 can perform 3D rendering according to the simulation data of a piece of clothes in the first action to obtain a first try-on effect, and perform 3D rendering according to the simulation data of the piece of clothes in the second action to obtain a second try-on effect. The first try-on effect and the second try-on effect can respectively represent the effect of the same 3D image wearing the same piece of clothes in the first action and the second action. Because the first action and the second action are different, the 3D image collides with the clothes in different actions, resulting in different deformations of the clothes. Therefore, the above-mentioned first try-on effect and the second try-on effect are different.

[0256] It can be seen that, Figure 5F and Figure 5E Therefore, Figure 5F the try-on action of the 3D image in the try-on effect display area 531 shown in FIG. 13B is different from Figure 5E the try-on action of the 3D image in the try-on effect display area 531 shown in FIG. 13A. Figure 5F The playing progress bar 533 shown in FIG. 13B is longer than that shown in FIG. 13A. That is, Figure 5E the playing progress of the try-on effect shown in FIG. 13B is more than that of the try-on effect shown in FIG. 13A. Figure 5F The playing progress of the try-on effect shown in FIG. 13B is more than that of the try-on effect shown in FIG. 13A. Figure 5E The playing progress of the try-on effect shown in FIG. 13B is more than that of the try-on effect shown in FIG. 13A. Similarly, the electronic device 100 can present the try-on effect of the 3D image wearing the clothes in different try-on actions. The user can view the deformation of the clothes when the 3D image performs different try-on actions.

[0257] It can be seen that, Figure 5G and Figure 5F Therefore, Figure 5G the try-on action of the 3D image in the try-on effect display area 531 shown in FIG. 13B is different from Figure 5F the try-on action of the 3D image in the try-on effect display area 531 shown in FIG. 13A. Figure 5G The playing progress bar 533 shown in FIG. 13B is longer than that shown in FIG. 13A. That is, Figure 5F the playing progress of the try-on effect shown in FIG. 13B is more than that of the try-on effect shown in FIG. 13A. Figure 5G The playing progress of the try-on effect shown in FIG. 13B is more than that of the try-on effect shown in FIG. 13A. Figure 5FThe playback progress of the virtual try-on effect is shown. Similarly, the electronic device 100 can display the virtual try-on effect of a 3D image wearing clothing and performing different try-on actions. Users can see the deformation of the clothing as the 3D image performs different try-on actions.

[0258] Understandably, the above Figures 5D-5G The content displayed in the fitting effect display area 531 can be a partial fitting effect diagram of a 3D figure wearing clothes and performing a continuous fitting action in the same scene.

[0259] like Figure 5G As shown, the scene option area 536 may include scene option 536A. In response to an operation on scene option 536A, the electronic device 100 can switch the virtual try-on scene from the scene corresponding to scene option 1 to the scene corresponding to scene option 536A.

[0260] like Figure 5H As shown, the electronic device 100 can display, in the fitting effect display area 531, the fitting effect of a 3D avatar wearing clothing and performing a fitting action in the scene corresponding to scene option 536A of scene 2. The selected scene option in scene option area 536 is determined by... Figure 5G The options for Scenario 1 shown have changed to Figure 5H The scenario shown is option 536A.

[0261] It can be seen that, Figures 5D-5G The scene presented in the fitting effect display area 531 shown is similar to... Figure 5H The scenes presented in the fitting effect display area 531 are different.

[0262] Understandably, since the 3D image, the clothing being tried on, and the fitting process remain unchanged, Figure 5H The 3D image and the clothing being tried on, displayed in the fitting effect display area 531, can be compared with... Figures 5D-5G The 3D image and the clothing being tried on are displayed consistently in the fitting effect display area 531. When only the scene is switched, the electronic device 100 can play the same 3D image wearing the same clothing and performing the same fitting action again in a new scene (such as the scene corresponding to scene 2 option 536A).

[0263] In some embodiments, the electronic device 100 can store preset body models. The user can directly use the existing body models in the electronic device 100 to virtually try on clothes without creating a 3D avatar. For example, the electronic device 100 can provide a male body model and a female body model. The user can select the male body model or the female body model to try on one or more clothes. The electronic device 100 can render a try-on effect of the selected body model wearing the selected clothes according to the selected clothes and the body model. Alternatively, the user can also select a try-on action and / or a try-on scene to view a try-on effect of the preset male body model or the preset female body model wearing the selected clothes to perform the selected try-on action in the specified try-on scene. The server 200 can store the body models consistent with the electronic device 100. The electronic device 100 can send the identification information (e.g., body model ID) of the selected body model to the server 200 and send the identification information (e.g., clothes ID) of the selected clothes to the server 200. The server 200 can perform cloth simulation according to the identification information of the body model and the identification information of the clothes to obtain simulation data of the clothes. The server 200 can send the simulation data of the clothes to the electronic device 100. Then, the electronic device 100 can render the try-on effect according to the received simulation data. Alternatively, the electronic device 100 can also send the 3D avatar data of the selected body model to the server 200. The method of virtually trying on clothes using the preset body model can help the user to refer to the upper body effect of the clothes to determine whether the clothes are suitable for the user.

[0264] As described above Figures 5A-5H The user can use his or her own 3D avatar or the 3D avatar of others to virtually try on clothes. During the virtual try-on, the user can adjust the clothes, the try-on action performed by the 3D avatar, and the try-on scene. The electronic device 100 can render a try-on effect of the 3D avatar wearing the clothes to perform the try-on action in the try-on scene according to the selected clothes, the try-on action, and the try-on scene to help the user accurately determine whether the clothes are suitable for the user corresponding to the 3D avatar. This can effectively reduce the situation that the user buys unsuitable clothes online and reduce the return rate of clothes e-commerce.

[0265] Figure 6 An exemplary flowchart of a virtual try-on method provided by an embodiment of the present application is shown.

[0266] As Figure 6 shown, the method can be applied to a communication system including the electronic device 100 and the server 200. Referring to the foregoing Figure 1The electronic device 100 may include a product display module 212, a 3D image management module 215, a 3D rendering module 216A, a digital clothing simulation SDK 216B, an action management module 216C, and a scene management module 216D. The server 200 may include a 3D digital clothing management module 220 and a 3D digital clothing simulation module 230. This virtual try-on method may include steps S611 to S622.

[0267] in:

[0268] 1. (S611~S615) Determine the clothing to be tried on, the trying-on action, and the trying-on scene.

[0269] S611, the product display module 212 can send a virtual try-on request to the 3D rendering module 216A.

[0270] In some embodiments, the product display module 212 can detect user actions involving virtual try-on of one or more garments. These user actions can be referred to the foregoing. Figure 5B The operation of the virtual try-on control 512 is shown. Then, the product display module 212 can send a virtual try-on request to the 3D rendering module 216A. This virtual try-on request can contain description information about the clothing. This description information can be used by the 3D rendering module 216A to determine which one or more garments are to be tried on.

[0271] Not limited to the product display module 212, the aforementioned virtual try-on request can also be sent by other modules. For example, the electronic device 100 may include a virtual try-on detection module. This module can detect the clothing selected by the user for virtual try-on. Then, the virtual try-on detection module can send the aforementioned virtual try-on request. The module that sends the virtual try-on request can be configured by the e-commerce application as needed.

[0272] Not limited to the aforementioned 3D rendering module 216A, the aforementioned virtual try-on request can also be received by other modules.

[0273] This application uses the example of the product display module 212 sending a virtual try-on request to the 3D rendering module 216A for illustration.

[0274] In some embodiments, in addition to the description information of the clothing, the virtual try-on request may also include the description information of the target user. The target user can be a user who needs to virtually try on clothing. For example, as... Figure 5CAs shown, a user can select a 3D avatar that needs to virtually try on clothes. The merchandise display module 212 can determine the target user according to the 3D avatar option in the user interface 520 that is in a selected state. The 3D avatar corresponding to the 3D avatar option in the selected state in the user interface 520 is the 3D avatar of the target user. The target information of the target user can be used by the 3D rendering module 216 to determine the target user of the virtual fitting.

[0275] S612, the 3D rendering module 216A can obtain the body data from the 3D avatar management module 215.

[0276] According to the target information of the target user in the virtual fitting request, the 3D rendering module 216 can obtain the body data of the target user.

[0277] In a possible implementation, the body data can be 3D avatar data of the target user. The 3D avatar data can include body mesh data and skeleton data. The body size information of the body mesh data can be the same as or similar to the real body size information of the target user. For example, the height, weight, shoulder width, chest circumference, waist circumference, hip circumference, and arm length of the body model constructed by the body mesh data can be the same as or similar to the real height, weight, shoulder width, chest circumference, waist circumference, hip circumference, and arm length of the target user. The body can be composed of multiple bones such as a body trunk, limbs, head, and the like. The skeleton data can be used to represent the topological structure of the skeleton of the target user, and can describe one or more actions of the whole body of the target user. The skeleton data of the target user can be used to correspond to the fitting action to determine the body mesh data of the 3D avatar of the target user under the fitting action.

[0278] In some embodiments, the 3D avatar data can also include face mesh information. The face mesh information can be used to make the 3D avatar have a more realistic and natural expression when displaying the virtual fitting effect. The face mesh information can be obtained by modeling the face of the target user.

[0279] In some embodiments, the 3D avatar data can also include hair mesh information. The hair mesh information can be used to show the effect of the hair of the target user during the virtual fitting. In this way, the user can know whether the clothes can match the hairstyle. The hair mesh information can be obtained by modeling the hair of the target user.

[0280] In some embodiments, the 3D avatar data can also include shoe mesh information. The shoe mesh information can be used to represent the virtual fitting effect of the 3D avatar wearing different shoes. In this way, the user can know whether the clothes can match the shoes. The shoe mesh information can be obtained by modeling the shoes of the target user.

[0281] The 3D image data can further include more or less content, such as mesh information of earrings, mesh information of a necklace, mesh information of a handbag, mesh information of sunglasses, and the like. Embodiments of the present application do not limit this.

[0282] In another possible implementation, the human body data can be base model weight information based on a set of human body base models. The human body base models can be preset human body models, which can correspond to base model data. The base model data can include base model mesh data and base model skeleton data. The server 200 can store base model data consistent with the electronic device 100 side. In this way, the server 200 can generate human body mesh data of the target user consistent with the electronic device 100 side according to the base model mesh data and the base model weight information of the target user, and generate skeleton data of the target user consistent with the electronic device 100 side according to the base model skeleton data and the base model weight information of the target user.

[0283] Without being limited to the base model weight information, the human body data can also be other parameters that can be used to generate human body mesh data of the target user. For example, a skinned multi person linear model (SMPL) parameter. The SMPL represents a parameterized human body model, which can express mesh information of a human body with a set of parameters (such as body shape parameters, pose parameters, and the like).

[0284] The data amount of the base model weight information or other parameters that can be used to generate human body mesh data of the target user is generally less than the data amount of the 3D image data. Therefore, the 3D rendering module 216A acquires the base model weight information or other parameters that can be used to generate human body mesh data of the target user, which can reduce the data transmission amount in subsequent steps S616 and S617, save the transmission bandwidth between the electronic device 100 and the server 200, and improve the efficiency of virtual fitting.

[0285] The parameters used to generate human body mesh data of the target user can be referred to as human body parameters. As known from the foregoing embodiments, the human body parameters can be, for example, base model weight information, or can be SMPL parameters, or can also be body size information (such as one or more of height, shoulder width, bust, waist, hip, arm circumference, arm length, neck circumference, abdominal circumference, hip circumference, thigh circumference, calf circumference, leg length, and the like). Embodiments of the present application do not limit the specific content of the human body parameters.

[0286] In some embodiments, the 3D avatar data of the target user can be obtained by human modeling and stored in the 3D avatar management module 215. The 3D avatar management module 215 can obtain the body size information of the user and generate the 3D avatar data such as the human mesh data and the skeleton data of the user according to the body size information. Alternatively, the 3D avatar management module 215 can generate the 3D avatar data of the user by traditional manual modeling or scanning modeling. For example, the 3D avatar management module 215 can obtain one or more images containing the human body of the user. The 3D avatar management module 215 can process the one or more images by using image recognition methods such as human key point recognition to generate the 3D avatar data such as the human mesh data and the skeleton data of the user. The method of generating the 3D avatar data by human modeling is only an exemplary description of the present application. The 3D avatar management module 215 can also use other methods to construct a human model.

[0287] Optionally, the 3D avatar management module 215 can use the 3D avatar data obtained by human modeling and the base model data to generate the base model weight information.

[0288] S613, the 3D rendering module 216A can obtain the clothing data from the 3D digital clothing management module 220.

[0289] According to the description information of the clothing in the virtual fitting request, the 3D rendering module 216A can obtain the clothing data of the clothing.

[0290] In a possible implementation, the clothing data obtained by the 3D rendering module 216A can include 3D digital clothing data of the clothing to be tried on. The 3D digital clothing data can include clothing mesh data and clothing parameter information.

[0291] The clothing mesh data can include multiple clothing sub-mesh data. For example, the clothing sub-mesh data can include material mesh data, part mesh data, and the like. The material mesh data can be used to indicate the material of the clothing (such as silk, cotton, leather, and the like). The part mesh data can be used to indicate the mesh data of a part of the clothing (such as a sleeve, a collar, and the like). Alternatively, the clothing sub-mesh data can include mesh data of a combination of the material and the part of the clothing.

[0292] The clothing parameter information can include one or more of the following: a stretching coefficient, a bending coefficient, a friction coefficient, and the like. The clothing parameter information can be determined according to the physical material of the cloth of the clothing. In some embodiments, the clothing parameter information can also include connection information between different clothing sub-mesh data, for example, bending direction information at the connection of different parts of the clothing, and the like.

[0293] In another possible implementation, the clothing data acquired by the 3D rendering module 216A can be identification information of the clothing being tried on, such as a clothing ID. This clothing identification information can be used by the server 200 to determine the 3D digital clothing data corresponding to the clothing ID. The amount of data for the clothing identification information is much smaller than the amount of data for the 3D digital clothing data. Therefore, transmitting the clothing identification information between the electronic device 100 and the server 200 can reduce the amount of data transmitted and save transmission bandwidth between the electronic device 100 and the server 200.

[0294] Among them, the same garment with different sizes (such as S, M, L, etc.) can have different garment IDs, or can be distinguished by a combination of garment ID and size code. This can more realistically simulate the fitting effect of different sizes of clothing on the same 3D avatar, so that users can choose the appropriate size of clothing.

[0295] The 3D digital clothing data for the same garment can be different even if they are different sizes.

[0296] In some embodiments, the garment being tried on can be a single garment or a combination of multiple garments. In the case of virtually trying on multiple garments, the 3D rendering module 216A can also acquire the layer information of these multiple garments. This layer information can represent the overlapping layers of these garments in the 3D image. That is, the garment data can also include layer information of the garments. For example, in... Figure 5D In the virtual try-on scenario of a shirt and skirt, the shirt and skirt are layered differently. The shirt can be layered over the skirt, partially obscuring it. Similarly, in the virtual try-on scenario of a shirt and jacket, the jacket and skirt are layered differently. The jacket can be layered over the shirt, partially obscuring it. This application does not limit the layering information of multiple garments in its embodiments.

[0297] The layering information for the aforementioned multiple garments can be preset or specified by the user. For example, a user can specify that the shirt is layered over the skirt, so that the shirt partially covers the skirt. Alternatively, a user can specify that the skirt is layered over the shirt, so that the skirt partially covers the shirt.

[0298] In some embodiments, after obtaining the clothing data of the clothing from the 3D digital clothing management module 220, the 3D rendering module 216A can store the clothing data. When the clothing data of the same clothing needs to be obtained again, the 3D rendering module 216A can read the clothing data from the memory of the electronic device 100. In this way, for the clothing data of the same clothing, the 3D rendering module 216A can not need to request the 3D digital clothing management module 220 to send the clothing data multiple times. This can reduce the amount of data transmitted between the electronic device 100 and the server 200, and improve the efficiency of virtual fitting.

[0299] In some embodiments, the above-mentioned 3D digital clothing data can be obtained through clothing modeling and stored in the server 200, for example, in the 3D digital clothing management module 220 of the server 200. The above-mentioned 3D digital clothing data can be modeled by professional digital clothing modeling software. Alternatively, the 3D digital clothing data can also be obtained by adding clothing parameter information in the mesh model output by a digital content creation (DCC) tool. The above-mentioned DCC tool can be a 3D modeling tool, for example, Maya, 3ds Max, Blender, etc. The embodiments of the present application do not limit the method of creating a clothing model and generating 3D digital clothing data.

[0300] In some embodiments, step S613 is optional. Wherein, consistent identification information (such as clothing ID) can be stored in the electronic device 100 and the server 200 for the same clothing. The 3D rendering module 216A can determine the identification information of the clothing according to the description information of the clothing in the virtual fitting request, and then send the identification information of the clothing to the 3D digital clothing simulation module 230 through the digital clothing simulation SDK 216B.

[0301] S614, the 3D rendering module 216A can obtain the action data from the action management module 216C.

[0302] The action data obtained by the 3D rendering module 216A can be the action data of the fitting action to be performed by the 3D avatar during virtual fitting.

[0303] In a possible implementation, the action data can include skeleton data. The above-mentioned skeleton data can describe one or more actions of the human body. Wherein, the skeleton data of one fitting action can represent the topology of the skeleton when the human body performs this one fitting action. The number of skeletons can determine the degree of detail of expressing the fitting action. The number of skeletons contained in the skeleton data representing one fitting action can be from a dozen to hundreds of skeletons. The number of skeletons can be determined according to the needs of the virtual fitting business. The embodiments of the present application do not limit the number of skeletons contained in the skeleton data of one fitting action.

[0304] Different fitting actions can correspond to different bone data. For example, a first action can correspond to first bone data. A second action can correspond to second bone data. In a case where the fitting actions that the 3D avatar needs to perform include multiple actions, the action data acquired by the 3D rendering module 216A can include bone data corresponding to multiple actions. For example, the fitting actions that the 3D avatar needs to perform include a first action and a second action. The action data acquired by the 3D rendering module 216A can include first bone data and second bone data.

[0305] In another possible implementation, the action data can include identification information of the fitting action, for example, an action ID. The server 200 can store identification information of the fitting action consistent with the electronic device 100 side. In this way, the server 200 can determine the fitting action according to the identification information of the fitting action, so that the cloth simulation can be performed according to the collision between the 3D avatar and the clothes in the fitting action, and the simulation data of the clothes is obtained.

[0306] The data amount of the identification information of the fitting action is much smaller than the data amount of the bone data. Therefore, the 3D rendering module 216A acquires the identification information of the fitting action can reduce the data transmission amount in subsequent steps S616 and S617, save the transmission bandwidth between the electronic device 100 and the server 200, and improve the efficiency of virtual fitting.

[0307] The above-mentioned action data can be action data of a preset fitting action, or can be action data of a fitting action selected by a user, or can be action data of a user action collected in real time through motion capture technology. The embodiments of the present application do not limit the specific content of the above-mentioned fitting action.

[0308] S615, the 3D rendering module 216A can acquire scene data from the scene management module 216D.

[0309] The above-mentioned scene data can include scene mesh data. The scene mesh data of a scene can be obtained by modeling the scene. The scene of virtual fitting can be, for example, a T stage, a stage, a square, a lawn, a beach, a house, and the like.

[0310] In some embodiments, the scene data can also include light information. The above-mentioned light information can be used by the 3D rendering module 216A to simulate the effects of sunny, cloudy or different stage light combinations when rendering.

[0311] In some embodiments, the scene data can also include particle information. The above-mentioned particle information can be used by the 3D rendering module 216A to simulate the effects of scenes such as rain or snow when rendering.

[0312] Embodiments of the present application do not limit the content contained in the scene data. The scene data can also contain more or less content.

[0313] The scene data of one or more scenes can be stored in the scene management module 216D, so that the user can select a fitting scene when virtually fitting. Providing the option of one or more scenes in the virtual fitting scene can increase the playability of the virtual fitting. The user can experience the virtual fitting effect in different scenes.

[0314] 2, (S616-S618) fabric simulation of the garment.

[0315] S616, the 3D rendering module 216A can send the body data, the action data, and the garment data to the digital garment simulation SDK 216B, and request to obtain the simulation data of the garment.

[0316] S617, the digital garment simulation SDK 216B can send a digital garment simulation request to the 3D digital garment simulation module 230.

[0317] The digital garment simulation SDK 216B can be used to be responsible for information interaction between the electronic device 100 and the server 200, and can code and decode data, send and receive data.

[0318] When receiving the request of the 3D rendering module 216 to obtain the simulation data of the garment, the digital garment simulation SDK 216B can send a digital garment simulation request to the 3D digital garment simulation module 230. The digital garment simulation request can contain the body data, the action data, and the garment data in step S616. The digital garment simulation SDK 216B can compress and encode the body data, the action data, and the garment data before sending them to the 3D digital garment simulation module 230. This can reduce the data transmission amount between the electronic device 100 and the server 200, and save the transmission bandwidth.

[0319] In some embodiments, the digital garment simulation SDK 216B can also provide identity authentication and authentication functions.

[0320] S618, the 3D digital garment simulation module 230 can perform digital garment simulation to obtain simulation data.

[0321] In a possible implementation, the 3D digital garment simulation module 230 can calculate the body mesh data of the target user in one or more actions according to the body data and the action data in the digital garment simulation request, using a bone skinning algorithm. The one or more actions are the fitting actions that the 3D image of the target user needs to perform in the virtual fitting.

[0322] The 3D digital clothing simulation module 230 can reuse one fitting action to the target user according to the skeleton data of the target user and the skeleton data of the one fitting action by using an action reorientation method, so as to obtain the skeleton data of the target user under the one fitting action. The action reorientation can reuse the same fitting action to different target users, and can prevent the 3D image of each target user from losing the proportion or producing unnecessary deformation when performing the same fitting action.

[0323] Then, the 3D digital clothing simulation module 230 can generate the human mesh deformation data of the target user according to the human mesh data of the target user and the skeleton data of the target user under one fitting action by using a skeleton skinning algorithm, and further determine the human mesh data of the target user under the one fitting action. That is, the deformation of the human model of the target user driven by the skeleton data of the fitting action can be determined by the skeleton skinning algorithm.

[0324] Further, the 3D digital clothing simulation module 230 can determine the simulation data of the clothing after the 3D image of the target user collides with the clothing under one or more actions according to the human mesh data of the target user under the one or more actions and the clothing data in the digital clothing simulation request. The simulation data of the clothing can include clothing mesh data. The clothing mesh data can be the mesh data after the deformation of the clothing colliding with the 3D image of the target user, and can reflect the shape (which can include deformation details such as wrinkles on the clothing) after the collision of the clothing and the 3D image of the target user performing the fitting action.

[0325] In some embodiments, the action reorientation according to the skeleton data of the target user and the skeleton data of the fitting action is optional. The skeleton data of the fitting action can be used as the skeleton data of the target user to drive the human mesh of the target user. Specifically, the 3D digital clothing simulation module 230 can obtain the 3D image data of the target user according to the received human data. The 3D image data can include the human mesh data of the target user. The 3D digital clothing simulation module 230 can generate the human mesh deformation data of the target user according to the human mesh data of the target user and the skeleton data of the fitting action by using a skeleton skinning algorithm, and further determine the human mesh data of the target user under the one fitting action. That is, the 3D image data can also not include the skeleton data of the user.

[0326] The above embodiments can reduce the amount of calculation in creating a 3D figure and a cloth simulation process. Moreover, when the information of the arm length, leg length, body length, etc. corresponding to the skeleton data of the fitting action is relatively close to the actual arm length, leg length, body length, etc. of the target user, the human body mesh data of the target user performing the fitting action can be obtained more accurately, and the virtual fitting effect rendered can more realistically reflect the effect of the target user wearing the clothes.

[0327] It can be understood that, in the case that the 3D figure needs to perform multiple fitting actions, the simulation data of the clothes can include a clothes mesh sequence. The clothes mesh sequence can include multiple pieces of clothes mesh data. One piece of clothes mesh data can correspond to one fitting action, i.e., can reflect the shape after the clothes and the 3D figure of the target user collide when performing one fitting action.

[0328] It should be noted that, if the human body data in the digital clothes simulation request is base model weight information, the 3D digital clothes simulation module 230 can obtain the base model data from the 3D digital clothes management module 220, and determine the human body mesh data of the target user according to the base model mesh data in the base model data and the base model weight information, and determine the skeleton data of the target user according to the base model skeleton data in the base model data and the base model weight information. Alternatively, the 3D digital clothes simulation module 230 sends the base model weight information to the 3D digital clothes management module 220. The 3D digital clothes management module 220 can store the base model data. The 3D digital clothes management module 220 can determine the 3D figure data of the target user according to the base model data and the base model weight information, and send the 3D figure data of the target user to the 3D digital clothes simulation module 230. In this way, the 3D digital clothes simulation module 230 can perform cloth simulation according to the 3D figure data of the target user to obtain the simulation data of the clothes.

[0329] If the clothes data in the digital clothes simulation request is identification information of the clothes, the 3D digital clothes simulation module 230 can obtain the 3D digital clothes data of the clothes from the 3D digital clothes management module 220 according to the identification information of the clothes, so as to use the 3D digital clothes data to perform cloth simulation.

[0330] If the action data in the digital clothes simulation request is identification information of the fitting action, the 3D digital clothes simulation module 230 can obtain the skeleton data of the fitting action from the 3D digital clothes management module 220 according to the identification information of the fitting action, so as to use the skeleton data of the fitting action to perform cloth simulation.

[0331] 3、(S619-S622) rendering the fitting effect.

[0332] S619, the 3D digital clothing simulation module 230 can send the simulation data to the digital clothing simulation SDK 216B.

[0333] S620, the digital clothing simulation SDK 216B can preprocess the simulation data.

[0334] The preprocessing can include calculating Normal data corresponding to the clothing mesh data in the simulation data, so as to optimize the rendering effect of the 3D rendering module 216A in the subsequent step S622. The specific processing process of the preprocessing is not limited in the embodiments of the present application.

[0335] S621, the digital clothing simulation SDK 216B can send the preprocessed simulation data to the 3D rendering module 216A.

[0336] In a possible implementation, the step S620 is optional. That is, the preprocessing can not be completed by the digital clothing simulation SDK 216B.

[0337] For example, the preprocessing can be completed by the 3D rendering module 216A. The digital clothing simulation SDK 216B can send the received simulation data to the 3D rendering module 216A. The 3D rendering module 216A pre-processes the simulation data, and then performs the step S622.

[0338] For another example, the preprocessing can be completed by the 3D digital clothing simulation module 230. After the 3D digital clothing simulation module 230 performs cloth simulation to obtain the simulation data of the clothing, the 3D digital clothing simulation module 230 can preprocess the simulation data. Then, the 3D digital clothing simulation module 230 can send the preprocessed simulation data to the digital clothing simulation SDK 216B. The digital clothing simulation SDK 216B sends the preprocessed simulation data to the 3D rendering module 216A.

[0339] S622, the 3D rendering module 216A can complete rendering according to the 3D image data, the action data, the scene data and the preprocessed simulation data.

[0340] In some embodiments, in the process of virtual fitting, the 3D image of the target user needs to perform a plurality of fitting actions. The human body mesh data of the 3D image of the target user is different when performing different fitting actions, and the simulation data of the clothing is also different. Each fitting action can correspond to human body mesh data and simulation data of the clothing. After the 3D rendering module 216A receives the preprocessed simulation data, the 3D rendering module 216A can synchronize the human body mesh data and the preprocessed simulation data, so as to ensure that the 3D image can wear the clothing constructed by the simulation data corresponding to one fitting action when performing the one fitting action, and reduce the problem of clothing fitting.

[0341] Optionally, the data synchronization described above can also be completed by the digital garment simulation SDK 216B. That is, the pre-processing of the simulation data by the digital garment simulation SDK 216B can further include data synchronization of the human body mesh data and the simulation data. The digital garment simulation SDK 216B can send information of the data synchronization to the 3D rendering module 216A. When rendering, the 3D rendering module 216A can combine the human body mesh data corresponding to one fitting action and the simulation data of the garment according to the information of the data synchronization. The data obtained after the pre-processing of the simulation data by the digital garment simulation SDK 216B can be referred to as pre-processed data. As known from the foregoing embodiments, the pre-processed data can include one or more of the following: Normal data obtained by performing Normal calculation on the simulation data of the garment, and information (i.e., data synchronization information) obtained by synchronizing the human body mesh data corresponding to one fitting action and the simulation data. The data synchronization information can be used to reduce the garment fitting problem when the 3D rendering module 216A renders the fitting effect. The present application does not limit the content of the pre-processed data described above. The pre-processed data can further include more or less content. The digital garment simulation SDK 216B can send the pre-processed data to the 3D rendering module 216A.

[0342] Optionally, the data synchronization described above can also be completed by the 3D digital garment simulation module 230. The 3D digital garment simulation module 230 can send synchronized simulation data and human body mesh data to the digital garment simulation SDK 216B. The synchronized simulation data and human body mesh data can represent simulation data and human body mesh data corresponding to the same fitting action. The digital garment simulation SDK 216B can further send the synchronized simulation data and human body mesh data to the 3D rendering module 216A.

[0343] The 3D rendering module 216A can call a rendering pipeline to render a fitting scene according to scene data, and render a fitting effect of the 3D figure wearing a garment to perform one or more fitting actions according to the 3D figure data, the action data, and the pre-processed simulation data.

[0344] In some embodiments, the 3D rendering module 216A can obtain preset lighting information and virtual camera perspective information, and complete rendering of the fitting effect under the condition of the preset lighting and virtual camera perspective. Optionally, the 3D rendering module 216A can further receive user interaction information and adjust the rendering content of the fitting effect.

[0345] For example, the user interaction information can be determined according to an operation of adjusting the virtual camera view angle by the user. The 3D rendering module 216A can render the fitting effect after the virtual camera view angle is adjusted. In this way, the user can manually adjust the viewing angle to view the virtual fitting effect in different angles.

[0346] For another example, the user interaction information can be determined according to an operation of adjusting the light by the user. The 3D rendering module 216A can render the fitting effect after the light is adjusted.

[0347] For another example, the user interaction information can be determined according to an operation of adjusting the fitting action. The 3D rendering module 216A can obtain the simulation data of the clothes after the fitting action is adjusted by the 3D digital clothes simulation module 230 through the digital clothes simulation SDK 216B, and then render the fitting effect of the 3D avatar wearing the clothes to perform the adjusted fitting action. The method of obtaining the simulation data of the clothes after the fitting action is adjusted by the 3D rendering module 216A can refer to the description of the foregoing steps S616-S621.

[0348] The embodiment of the present application does not limit the specific content of the user interaction information.

[0349] By Figure 6 It can be known that the virtual fitting method provided by the present application can realize high-precision virtual fitting effect on a mobile terminal with weak computing capability. Moreover, the mode of performing fabric simulation on the cloud side and performing rendering on the terminal side can reduce the computing cost of the server 200 and improve the concurrent capability of the server 200. The server 200 can simultaneously respond to the request of a plurality of mobile terminals for fabric simulation, and cooperates with the plurality of mobile terminals to provide virtual fitting service for a plurality of users. In this way, when the user purchases clothes online through the e-commerce application in the mobile terminal, the user can conveniently view the fitting effect of the 3D avatar wearing the clothes to perform the fitting action, so as to determine whether the clothes are suitable for the user. This can effectively reduce the situation that the user buys unsuitable clothes online, and reduce the return rate of the clothes e-commerce. In addition, since the human body model of the user and the clothes model of the clothes can be completed before virtual fitting, the electronic device 100 and the server 200 can directly use the established human body model and clothes model when cooperatively providing virtual fitting service. This can effectively improve the efficiency of virtual fitting, and the electronic device 100 can quickly present the virtual fitting effect to the user.

[0350] The following describes another communication system 70 provided by an embodiment of the present application.

[0351] Figure 7 An exemplary architecture diagram of the communication system 70 is shown. As shown in Figure 7 The communication system 70 can include an electronic device 710 and a server 720.

[0352] The electronic device 710 can include an action management module 711, a 3D figure management module 712, a digital clothing simulation SDK 713, a 3D rendering module 714, a scene management module 715. Among them, the action management module 711, the 3D figure management module 712, the digital clothing simulation SDK 713, the 3D rendering module 714, and the scene management module 715 can be respectively referred to the aforementioned descriptions of the action management module 216C, the 3D figure management module 215, the digital clothing simulation SDK 216B, the 3D rendering module 216A, and the scene management module 216D. Figure 1

[0353] The scene management module 715 is optional. In some embodiments, the scene management module 715 can also not be included in the electronic device 710.

[0354] The digital clothing simulation SDK 713 can include a codec module 713A, a network transmission module 713B, and a rendering preprocessing module 713C. The codec module 713A can be used for encoding data that needs to be sent, and decoding received data. The network transmission module 713B can be used for communication between the digital clothing simulation SDK 713 and the 3D digital clothing simulation module 724 in the server 720. The rendering preprocessing module 713C can be used for preprocessing the simulation data of the received clothing. For example, Normal calculation is performed on the simulation data of the clothing.

[0355] Among them, the digital clothing simulation SDK 713 can be referred to as a processing module of the electronic device 710.

[0356] The server 720 can include a 3D digital clothing management module 721, an action management module 722, a 3D figure management module 723, and a 3D digital clothing simulation module 724. Among them, the 3D digital clothing management module 721 and the 3D digital clothing simulation module 724 can be respectively referred to the aforementioned descriptions of the 3D digital clothing management module 220 and the 3D digital clothing simulation module 230. Figure 1

[0357] The 3D digital clothing simulation module 724 can include a cloth simulation module 724A, a codec module 724B, and a network transmission module 724C. The cloth simulation module 724A can be used for cloth simulation according to the 3D figure data, the skeleton data of the fitting action, and the 3D digital clothing data to obtain the simulation data of the clothing. The codec module 724B and the network transmission module 724C can be respectively referred to the aforementioned descriptions of the codec module 713A and the network transmission module 713B in the electronic device 710.

[0358] ​​The motion management module 722 can be used to determine the fitting action based on the fitting action ID or other motion identifier information, thereby obtaining the skeletal data of the fitting action. In one possible implementation, the motion management module 722 may store the fitting action ID consistent with that on the electronic device 710, as well as the skeletal data of the fitting action. The motion management module 722 can look up the skeletal data of the corresponding fitting action based on the fitting action ID.

[0359] The 3D image management module 723 can be used to generate human body mesh data consistent with that of the electronic device 710 based on human body mesh parameters.

[0360] In one possible implementation, the aforementioned human mesh parameters can be based on a set of human baseline template weight information. The 3D avatar management module 723 can store baseline template data consistent with that of the electronic device 710. The 3D avatar management module 723 generates 3D avatar data of the target user consistent with that of the electronic device 710 based on the baseline template data and the target user's baseline template weight information.

[0361] In another possible implementation, the aforementioned human mesh parameters may include the user's body size information (such as shoulder width, chest circumference, waist circumference, hip circumference, leg length, arm length, etc., one or more of which). The 3D image management module 723 may store a human mesh generation model consistent with the side of the electronic device 710. The 3D image management module 723 can generate human mesh data consistent with the side of the electronic device 710 based on the human mesh parameters and the mesh generation model.

[0362] In another possible implementation, the aforementioned human mesh parameters can be other parameters that can be used to generate human mesh data. For example, SMPL parameters, etc. This application does not limit the specific content of the aforementioned human mesh parameters.

[0363] like Figure 7 As shown, the digital clothing simulation SDK 713 in the electronic device 710 can obtain human mesh parameters from the 3D image management module 712 and the fitting action ID from the action management module 711. The digital clothing simulation SDK 713 can also obtain the clothing ID of the garment to be tried on. The digital clothing simulation SDK 713 can send the human mesh parameters, fitting action ID, and clothing ID to the 3D digital clothing simulation module 724 in the server 720. The human mesh parameters and fitting action ID are described above. The clothing ID can be used to identify the garment. When there are multiple garments to be tried on, the clothing ID sent by the digital clothing simulation SDK 713 can be the clothing IDs of multiple garments.

[0364] In some embodiments, the garment ID of a piece of garment can be a combination of the style ID of the piece of garment and the garment size information (e.g., S size, M size, L size, etc.). The style ID can be used to identify the style of the garment. It can be understood that the same style of garment can have different sizes. The same style of garment in different sizes usually has different effects when worn by a user. Thus, the user can adjust the size of the try-on when virtually trying on clothes, so as to find the most suitable size of the garment for himself.

[0365] In some embodiments, the garment ID of a piece of garment can be a combination of the manufacturer ID of the piece of garment, the style ID and the garment size information.

[0366] In some other embodiments, when the garment ID contains garment IDs of multiple pieces of garment, the garment ID can further contain the hierarchical information of the multiple pieces of garment.

[0367] The embodiments of the present application do not limit the specific content of the garment ID.

[0368] The 3D digital garment simulation module 724 can instruct the 3D figure management module 723 to determine the human mesh data by using the received human mesh parameter.

[0369] The 3D digital garment simulation module 724 can obtain the skeleton data of the try-on action from the action management module 722 by using the received try-on action ID. The try-on action can be a single action or an action sequence containing multiple actions.

[0370] The 3D digital garment simulation module 724 can obtain the 3D digital garment data (e.g., garment mesh, garment parameter information, etc.) of the garment from the 3D digital garment management module 721 by using the received garment ID.

[0371] The 3D digital garment simulation module 724 can restore the human mesh data corresponding to each try-on action by calculation by using the obtained human mesh data, the skeleton data of the try-on action and the 3D digital garment data. Further, the 3D digital garment simulation module 724 can perform cloth simulation to calculate the simulation data of the collision between the garment and the 3D figure after the try-on action. The simulation data of the garment can describe the deformation effects (e.g., bending and wrinkling) of the garment when the 3D figure wears the garment to perform the try-on action. The calculation of the cloth simulation can be real-time calculation (e.g., 30 frames per second), or non-real-time calculation (e.g., 1 frame per second), or super real-time calculation (e.g., 100 frames per second). The embodiments of the present application do not limit the rate of the calculation. The specific process of the cloth simulation can refer to the foregoing description of steps S616-S618. Figure 6

[0372] ​The 3D digital clothing simulation module 724 can send the clothing simulation data to the digital clothing simulation SDK 713 in the electronic device 710.

[0373] The digital clothing simulation SDK 713 can preprocess the simulation data of clothing and send the preprocessed simulation data to the 3D rendering module 714. The preprocessing can include calculating the normal information of each clothing mesh frame in the simulation data. The preprocessing can also include synchronizing the clothing simulation data with the human mesh of the 3D avatar during a fitting action. The simulation data and human mesh data corresponding to the same fitting action can be a single combination. In this way, the 3D rendering module 714 can render based on the synchronized data, ensuring that the 3D avatar can wear the clothing constructed from the simulation data corresponding to that fitting action when performing a fitting action, reducing clothing clipping issues.

[0374] Optionally, the digital clothing simulation SDK 713 can also send the received simulation data to the 3D rendering module 714. The 3D rendering module 714 can perform the aforementioned preprocessing on the simulation data, and then render the fitting effect based on the preprocessed simulation data. The process of the 3D rendering module 714 rendering the fitting effect can be referred to the aforementioned... Figure 6 The following is an introduction to steps S619 to S621.

[0375] It should be noted that the electronic device 710 may also include more or fewer modules. For example, the electronic device 710 may also include the aforementioned modules. Figure 1 The electronic device 100 shown includes modules such as account management 211, product display 212, order placement 213, payment 214, and scene management 216D. The server 720 may also contain more or fewer modules. For details on the server 720, please refer to the aforementioned description of the server 200.

[0376] From the above Figure 7 As shown in the communication system 70, during the virtual try-on process, the electronic device 710 can reduce data transmission volume by sending human mesh parameters, try-on action ID, and clothing ID to the server 720, saving transmission bandwidth between the electronic device 710 and the server 720 and improving the efficiency of virtual try-on. The electronic device 710 can provide high-precision virtual try-on effects. When users purchase clothing online using the electronic device 710, they can conveniently view the try-on effect of a 3D image matching their body shape wearing the clothing, thus determining whether the clothing suits them. This can effectively reduce the situation where users buy unsuitable clothing online, lowering the return rate of clothing e-commerce.

[0377] Another communication system 80 provided in the embodiments of this application is described below.

[0378] Figure 8 An exemplary architecture diagram of the communication system 80 is shown. As shown, the communication system 80 can include an electronic device 810 and a server 820. Figure 8

[0379] The electronic device 810 can include an action management module 811, a 3D figure management module 812, a digital clothing simulation SDK 813, a 3D rendering module 814, a scene management module 815. The digital clothing simulation SDK 813 can include a codec module 813A, a network transmission module 813B, a rendering preprocessing module 813C. The various modules included in the electronic device 810 can refer to the foregoing descriptions of the electronic device 710 shown in FIG. 7. Figure 7

[0380] The scene management module 815 is optional. In some embodiments, the electronic device 810 can also not include the scene management module 815.

[0381] The digital clothing simulation SDK 813 can be referred to as a processing module of the electronic device 810.

[0382] The server 820 can include a 3D digital clothing simulation module 821. The 3D digital clothing simulation module 821 can include a cloth simulation module 821A, a codec module 821B, a network transmission module 821C. The 3D digital clothing simulation module 821 can refer to the foregoing descriptions of the 3D digital clothing simulation module 724 shown in FIG. 7. Figure 7

[0383] In some embodiments, the digital clothing simulation SDK 813 can obtain 3D figure data from the 3D figure management module 812, and obtain skeleton data of a fitting action from the action management module 811. The digital clothing simulation SDK 813 can also obtain 3D digital clothing data of a clothing to be fitted. The 3D digital clothing data can be obtained by the digital clothing simulation SDK 813 from the server 820, or can be obtained from a server associated with an e-commerce application, or can be obtained from a memory of the electronic device 810.

[0384] As shown, the digital clothing simulation SDK 813 can send the 3D figure data, the skeleton data of the fitting action, and the 3D digital clothing data to the 3D digital clothing simulation module 821 in the server 820. Figure 8

[0385] ​​​​The 3D digital clothing simulation module 821 can perform fabric simulation based on the received 3D image data, skeletal data of the fitting action, and 3D digital clothing data to obtain clothing simulation data. During the fabric simulation process, the 3D digital clothing simulation module 821 can also calculate the human mesh data of the user's 3D image during the fitting action. The fabric simulation process described above can be found in the preceding embodiments and will not be repeated here.

[0386] The 3D digital clothing simulation module 821 can send the clothing simulation data and the human mesh data of the person performing the fitting action to the digital clothing simulation SDK 813. In this way, the digital clothing simulation SDK 813 or the 3D rendering module 814 does not need to synchronize data. The 3D rendering module 814 can use the aforementioned simulation data and the human mesh data of the person performing the fitting action to render the fitting effect. The implementation of the fitting effect rendering by the 3D rendering module 814 can be referred to the description in the foregoing embodiments.

[0387] In some embodiments, the normal calculation of the simulation data can be performed by the server 820. This reduces the computational power consumption on the electronic device 810 side. Alternatively, the normal calculation of the simulation data can be performed by the electronic device 810. This reduces the transmission bandwidth between the electronic device 810 and the server 820.

[0388] It should be noted that the electronic device 810 may also include more or fewer modules. For example, the electronic device 810 may also include the aforementioned modules. Figure 1 The electronic device 100 shown includes modules such as account management 211, product display 212, order placement 213, payment 214, and scene management 216D. The server 820 may also contain more or fewer modules. For details on the server 820, please refer to the aforementioned description of the server 200.

[0389] In some embodiments, the above Figure 8 The communication system 80 shown can be applied to scenarios where there is insufficient trust between the electronic device 810 and the server 820. For example, even if the server 820 does not include a 3D image management module, an action management module, or a 3D digital clothing management module, the electronic device 810 and the server 820 can collaborate on an end-to-cloud basis using the communication system 80 to provide users with virtual try-on services.

[0390] Another communication system 90 provided in the embodiments of this application is described below.

[0391] Figure 9 An exemplary architecture diagram of a communication system 90 is shown. Figure 9 The communication system 90 may include an electronic device 910 and a server 920.

[0392] The electronic device 910 can include an action management module 911, a 3D figure management module 912, a digital clothing simulation SDK 913, a 3D rendering module 914, and a scene management module 915. The digital clothing simulation SDK 913 can include a codec module 913A, a network transmission module 913B, and a rendering preprocessing module 913C. The various modules included in the electronic device 910 can be referred to the foregoing descriptions of the various modules included in the electronic device 710 shown in FIG. 7. Figure 7 The electronic device 910 can include an action management module 911, a 3D figure management module 912, a digital clothing simulation SDK 913, a 3D rendering module 914, and a scene management module 915. The digital clothing simulation SDK 913 can include a codec module 913A, a network transmission module 913B, and a rendering preprocessing module 913C. The various modules included in the electronic device 910 can be referred to the foregoing descriptions of the various modules included in the electronic device 710 shown in FIG. 7.

[0393] The scene management module 915 is optional. In some embodiments, the electronic device 910 can also not include the scene management module 915.

[0394] The digital clothing simulation SDK 913 can be referred to as a processing module of the electronic device 910.

[0395] The server 920 can include a 3D digital clothing management module 921, an action management module 922, a 3D figure management module 923, a 3D digital clothing simulation module 924, and a database 925. The 3D digital clothing simulation module 924 can include a cloth simulation module 924A, a codec module 924B, and a network transmission module 924C.

[0396] The database 925 can be used to store simulation data of a clothing in a 3D figure + clothing + fitting action combination.

[0397] In some embodiments, the 3D digital clothing simulation module 924 can store the simulation data of the clothing obtained through cloth simulation to the database 925, and record the combination of the 3D figure + clothing + fitting action corresponding to the simulation data in the database.

[0398] In the process of virtual fitting, the digital clothing simulation SDK 913 can send the human mesh parameter, the fitting action ID, and the clothing ID to the 3D digital clothing simulation module 924. According to the received human mesh parameter, the fitting action ID, and the clothing ID, the 3D digital clothing simulation module 924 can search whether there is a same or similar 3D figure + clothing + fitting action combination in the database 925. The same 3D figure + clothing + fitting action combination can represent the same 3D figure (such as the same body size information of the 3D figure), the same clothing (such as the same style and size of the clothing), and the same fitting action combination. The similar 3D figure + clothing + fitting action combination can represent the similar 3D figure (such as the difference between the body size information of the 3D figure is less than a preset threshold), the same clothing, and the same fitting action combination.

[0399] When the same or similar 3D figure + clothes + fitting action combination is not found from the database 925, the 3D digital clothes simulation module 924 can perform cloth simulation by using the received human body mesh parameters, fitting action ID, and clothes ID, to obtain simulation data of the clothes. The implementation method of the cloth simulation can refer to the foregoing description of the embodiment shown in Figure 7 After that, the 3D digital clothes simulation module 924 can send the simulation data of the clothes to the digital clothes simulation SDK 913.

[0400] When the same or similar 3D figure + clothes + fitting action combination is found from the database 925, the 3D digital clothes simulation module 924 can obtain historical simulation data (i.e., simulation data of the clothes corresponding to the same or similar 3D figure + clothes + fitting action combination) from the database 925. The 3D digital clothes simulation module 924 can send the historical simulation data to the digital clothes simulation SDK 913.

[0401] After the electronic device 910 receives the simulation data of the clothes from the server 920, the virtual fitting effect can be rendered. For details, refer to the foregoing description of the embodiment, which will not be repeated here.

[0402] It should be noted that the electronic device 910 can further include more or fewer modules. For example, the electronic device 910 can further include the account management module 211, the commodity display module 212, the order purchase module 213, the payment module 214, the scene management module 216D, and the like in the electronic device 100 shown in Figure 1 The server 920 can also include more or fewer modules. The server 920 can refer to the foregoing description of the server 200.

[0403] As known from the communication system 90 shown in Figure 9 The server 920 can store the simulation data of the clothes obtained by cloth simulation. When the same fitting action is needed to be performed on the same 3D figure to put on the same clothes for cloth simulation again, the server 920 can obtain the simulation data of the clothes from the database 925, without the need to perform the specific calculation process of cloth simulation again. In this way, the server 920 can reduce the repeated cloth simulation process, save the computing resources of the server 920, and improve the concurrency capability of the server 920.

[0404] It can be understood that the various user interfaces described in the embodiments of the present application are only example interfaces, and do not limit the present application. In other embodiments, the user interface can have a different interface layout, can include more or fewer controls, and can increase or decrease other function options, as long as the same invention idea is provided based on the present application, which is within the protection scope of the present application.

[0405] It should be noted that any feature in any embodiment of the present application, or any part of any feature, can be combined with any other feature in any embodiment of the present application, or any part of any feature, without departing from the scope of the present application.

[0406] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual fitting method, characterized by, The method comprises: The electronic device receives a first operation for selecting a first garment for virtual fitting; In response to the first operation, the electronic device sends first body data of a first 3D avatar, first action data of a first action sequence, and first garment data of the first garment to a server, wherein the first action sequence comprises a first action and a second action; The electronic device receives first simulation data of the first garment from the server; The electronic device renders a first set of fitting effects according to the first simulation data, wherein the first set of fitting effects comprises a first fitting effect and a second fitting effect, the first fitting effect represents an effect of the first 3D avatar wearing the first garment to perform the first action, and the second fitting effect represents an effect of the first 3D avatar wearing the first garment to perform the second action, and the first fitting effect and the second fitting effect are different; The electronic device receives a third operation for selecting a second garment for virtual fitting; In response to the third operation, the electronic device sends second garment data of the second garment and first hierarchical information to the server, wherein the first hierarchical information indicates a stacking order of the first garment and the second garment; The electronic device receives second simulation data from the server; The electronic device renders a second set of fitting effects according to the second simulation data, wherein the second set of fitting effects comprises a third fitting effect and a fourth fitting effect, the third fitting effect represents an effect of the first 3D avatar wearing the first garment and the second garment to perform the first action, and the fourth fitting effect represents an effect of the first 3D avatar wearing the first garment and the second garment to perform the second action.

2. The method of claim 1, wherein, The first body data comprises first body mesh data of the first 3D avatar, the first action data comprises first skeleton data of the first action and second skeleton data of the second action, and the first garment data comprises garment mesh data and garment parameter information of the first garment, wherein the garment parameter information comprises one or more of the following: a stretch coefficient, a bending coefficient, and a friction coefficient of the first garment. Alternatively, The first body data comprises body parameters of the first 3D avatar, the body parameters are used to generate the first body mesh data, the first action data comprises identification information of the first action and the second action, and the first garment data comprises identification information of the first garment.

3. The method according to claim 1 or 2, characterized in that, Before the electronic device sends first body data of a first 3D avatar, first action data of a first action sequence, and first garment data of the first garment to the server, the method further comprises: The electronic device receives an operation for selecting the first 3D avatar and / or the first action sequence.

4. The method according to claim 1 or 2, characterized in that, The first set of fitting effects is presented by a first video playback, and the method further comprises: The electronic device receives a second operation for the first video, the second operation being for playing the first video, or for pausing playing the first video, or for fast forwarding the first video, or for fast rewinding the first video; The electronic device controls playing the first video according to the second operation.

5. The method of claim 3, wherein, The first set of try-on effects is presented by the first video playing, and the method further comprises: The electronic device receives a second operation for the first video, the second operation being for playing the first video, or for pausing playing the first video, or for fast forwarding the first video, or for fast rewinding the first video; The electronic device controls playing the first video according to the second operation.

6. The method of any one of claims 1, 2, 5, wherein, The method further comprises: The electronic device receives a fourth operation for selecting a third garment to replace the first garment for virtual try-on; In response to the fourth operation, the electronic device sends third garment data of the third garment to the server; The electronic device receives third simulation data of the third garment from the server; The electronic device renders a third set of try-on effects according to the third simulation data, the third set of try-on effects comprising a fifth try-on effect and a sixth try-on effect, the fifth try-on effect being for representing an effect of the first 3D avatar wearing the third garment to perform the first action, and the sixth try-on effect being for representing an effect of the first 3D avatar wearing the third garment to perform the second action.

7. The method of claim 3, wherein, The method further comprises: The electronic device receives a fourth operation for selecting a third garment to replace the first garment for virtual try-on; In response to the fourth operation, the electronic device sends third garment data of the third garment to the server; The electronic device receives third simulation data of the third garment from the server; The electronic device renders a third set of try-on effects according to the third simulation data, the third set of try-on effects comprising a fifth try-on effect and a sixth try-on effect, the fifth try-on effect being for representing an effect of the first 3D avatar wearing the third garment to perform the first action, and the sixth try-on effect being for representing an effect of the first 3D avatar wearing the third garment to perform the second action.

8. The method of claim 4, wherein, The method further comprises: The electronic device receives a fourth operation for selecting a third garment to replace the first garment for virtual try-on; In response to the fourth operation, the electronic device sends third garment data of the third garment to the server; The electronic device receives third simulation data of the third garment from the server; The electronic device renders a third set of try-on effects according to the third simulation data, the third set of try-on effects comprising a fifth try-on effect and a sixth try-on effect, the fifth try-on effect being for representing an effect of the first 3D avatar wearing the third garment to perform the first action, and the sixth try-on effect being for representing an effect of the first 3D avatar wearing the third garment to perform the second action.

9. The method of any one of claims 1, 2, 5, wherein, The method further includes: The electronic device receives a fifth operation for changing the first action sequence into a second action sequence, the second action sequence including a third action and a fourth action; In response to the fifth operation, the electronic device sends second action data of the second action sequence to the server; The electronic device receives fourth simulation data of the first garment from the server; The electronic device renders a fourth set of fitting effects according to the fourth simulation data, the fourth set of fitting effects including a seventh fitting effect and an eighth fitting effect, the seventh fitting effect being used to represent an effect of the first 3D image wearing the first garment to perform the third action, and the eighth fitting effect being used to represent an effect of the first 3D image wearing the first garment to perform the fourth action.

10. The method of claim 3, wherein, The method further includes: The electronic device receives a fifth operation for changing the first action sequence into a second action sequence, the second action sequence including a third action and a fourth action; In response to the fifth operation, the electronic device sends second action data of the second action sequence to the server; The electronic device receives fourth simulation data of the first garment from the server; The electronic device renders a fourth set of fitting effects according to the fourth simulation data, the fourth set of fitting effects including a seventh fitting effect and an eighth fitting effect, the seventh fitting effect being used to represent an effect of the first 3D image wearing the first garment to perform the third action, and the eighth fitting effect being used to represent an effect of the first 3D image wearing the first garment to perform the fourth action.

11. The method of claim 4, wherein, The method further includes: The electronic device receives a fifth operation for changing the first action sequence into a second action sequence, the second action sequence including a third action and a fourth action; In response to the fifth operation, the electronic device sends second action data of the second action sequence to the server; The electronic device receives fourth simulation data of the first garment from the server; The electronic device renders a fourth set of fitting effects according to the fourth simulation data, the fourth set of fitting effects including a seventh fitting effect and an eighth fitting effect, the seventh fitting effect being used to represent an effect of the first 3D image wearing the first garment to perform the third action, and the eighth fitting effect being used to represent an effect of the first 3D image wearing the first garment to perform the fourth action.

12. The method of any one of claims 1, 2, 5, wherein, The method further includes: The electronic device receives a sixth operation for selecting a fitting scene as a first scene; In response to the sixth operation, the electronic device obtains first scene data of the first scene; The electronic device renders a first set of fitting effects according to the first simulation data, specifically including: The electronic device renders the first set of try-on effects according to the first scene data and the first simulation data, the first set of try-on effects including the first try-on effect and the second try-on effect, the first try-on effect being used to represent an effect that the first 3D image wears the first clothes to perform the first action in the first scene, and the second try-on effect being used to represent an effect that the first 3D image wears the first clothes to perform the second action in the first scene.

13. The method of claim 3, wherein, The method further includes: The electronic device receives a sixth operation for selecting a try-on scene as the first scene; In response to the sixth operation, the electronic device acquires first scene data of the first scene; The electronic device renders the first set of try-on effects according to the first simulation data, specifically including: The electronic device renders the first set of try-on effects according to the first scene data and the first simulation data, the first set of try-on effects including the first try-on effect and the second try-on effect, the first try-on effect being used to represent an effect that the first 3D image wears the first clothes to perform the first action in the first scene, and the second try-on effect being used to represent an effect that the first 3D image wears the first clothes to perform the second action in the first scene.

14. The method of claim 4, wherein, The method further includes: The electronic device receives a sixth operation for selecting a try-on scene as the first scene; In response to the sixth operation, the electronic device acquires first scene data of the first scene; The electronic device renders the first set of try-on effects according to the first simulation data, specifically including: The electronic device renders the first set of try-on effects according to the first scene data and the first simulation data, the first set of try-on effects including the first try-on effect and the second try-on effect, the first try-on effect being used to represent an effect that the first 3D image wears the first clothes to perform the first action in the first scene, and the second try-on effect being used to represent an effect that the first 3D image wears the first clothes to perform the second action in the first scene.

15. The method of claim 2, wherein, The method further includes: The electronic device acquires first body size information, the first body size information including one or more of the following: height, shoulder width, bust, waist, hip, arm circumference, arm length, neck circumference, abdominal circumference, crotch circumference, thigh circumference, calf circumference, leg length; The electronic device obtains first human body mesh data of the first 3D image according to the first body size information; Or, The electronic device acquires one or more images, the one or more images displaying a human body corresponding to the first 3D image; The electronic device obtains first human body mesh data of the first 3D image according to the one or more images.

16. A virtual fitting method, characterized by, The method is performed by a processing module, and the method includes: obtaining first body data of a first 3D image, first action data of a first action sequence, and first clothing data of a first clothing, wherein the first action sequence comprises a first action and a second action; sending the first body data, the first action data, and the first clothing data to a server; receiving first simulation data of the first clothing from the server; rendering a first set of fitting effects from the first simulation data by a 3D rendering module, wherein the first set of fitting effects comprises a first fitting effect and a second fitting effect, the first fitting effect is used to show an effect of the first 3D image wearing the first clothing to perform the first action, and the second fitting effect is used to show an effect of the first 3D image wearing the first clothing to perform the second action, and the first fitting effect and the second fitting effect are different; obtaining second clothing data of a second clothing and first hierarchical information, wherein the first hierarchical information is used to indicate a wearing order of the first clothing and the second clothing; sending the second clothing data and the first hierarchical information to the server; receiving second simulation data from the server; rendering a second set of fitting effects from the second simulation data by the 3D rendering module, wherein the second set of fitting effects comprises a third fitting effect and a fourth fitting effect, the third fitting effect is used to show an effect of the first 3D image wearing the first clothing and the second clothing to perform the first action, and the fourth fitting effect is used to show an effect of the first 3D image wearing the first clothing and the second clothing to perform the second action.

17. The method of claim 16, wherein, The first body data comprises first body mesh data of the first 3D image, the first action data comprises first skeleton data of the first action and second skeleton data of the second action, the first clothing data comprises clothing mesh data and clothing parameter information of the first clothing, and the clothing parameter information comprises one or more of the following: a stretching coefficient, a bending coefficient, and a friction coefficient of the first clothing. Alternatively, The first body data comprises body parameters of the first 3D image, the body parameters are used to generate the first body mesh data, the first action data comprises identification information of the first action and the second action, and the first clothing data comprises identification information of the first clothing.

18. The method according to claim 16 or 17, characterized in that, The rendering of the first set of fitting effects from the first simulation data by the 3D rendering module specifically comprises: preprocessing the first simulation data to obtain preprocessing data, wherein the preprocessing comprises one or more of the following: normal calculation on the first simulation data, and data synchronization on the first simulation data and the first 3D image with the first body data of the first action sequence; rendering the first set of fitting effects from the preprocessing data by the 3D rendering module.

19. An electronic device, comprising: The electronic device comprises a communication device, a memory, and a processor, wherein the communication device is used to communicate with a server; the memory is used to store a computer program; and the processor is used to execute the computer program. The processor is configured to invoke the computer program to cause the electronic device to perform the method of any one of claims 1-15.

20. A computer readable storage medium comprising instructions, wherein: The instructions, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1-15.

21. A computer program product, characterised in that, The computer program product comprises computer instructions which, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1-15.

Images