Virtual fitting method, related device and communication system

Abstract

The invention provides a virtual fitting method, a related device and a communication system. The mobile terminal can respond to clothes selection and fitting actions of a user to perform virtual fitting operation, and 3D image data, 3D digital clothes data and action data of the user are sent to the server. The server can solve the clothes deformation information after the human body collides with the clothes under the corresponding fitting action through the physical simulation technology, and the simulation data of the clothes are obtained. The mobile terminal can render the fitting effect that the 3D image of the user wears the clothes to perform the fitting action according to the simulation data sent by the server. The fitting effect can vividly present clothing information such as the style and the cloth of the clothing, and deformation information of details such as wrinkles generated by collision of the 3D image and the digital clothing. Therefore, the user can be helped to know the upper body effect of the clothing before ordering and purchasing on the online clothing market, the situation that improper clothing is bought on line is reduced, and the return rate of clothing e-commerce is reduced.

Description

[0001] This application is a divisional application. The original application has the application number 202310491407.8 and the original application date is April 28, 2023. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to virtual try-on methods, related devices and communication systems. Background Technology

[0003] With the development of the online apparel market, more and more users are buying clothes online through e-commerce platforms. However, users cannot try on clothes before purchasing them online, often resulting in unsuitable items that need to be returned. This leads to a poor user shopping experience and a high return rate for apparel e-commerce platforms. Summary of the Invention

[0004] This application provides a virtual try-on method, related devices, and a communication system. The electronic device can render a 3D avatar of the user wearing the clothing based on the user's body data, the clothing selected by the user, and the user's try-on actions. The try-on effect can realistically present the style, fabric, and other clothing information, as well as the deformation information such as wrinkles caused by the collision between the clothing and the 3D avatar. Specifically, different deformations can occur when the 3D avatar performs different try-on actions. The electronic device can render different try-on effects corresponding to different try-on actions performed by the 3D avatar, presenting the user with the dynamic changes of the clothing during the different try-on actions of the 3D avatar.

[0005] In one aspect, this application provides a virtual try-on method. An electronic device receives a first operation to select a first garment for virtual try-on; in response to the first operation, the electronic device sends to a server first human body data of a first 3D avatar, first action data of a first action sequence, and first garment data of a first garment, wherein the first action sequence includes a first action and a second action; the electronic device receives first simulation data of the first garment from the server; the electronic device renders a first set of try-on effects based on the first simulation data, the first set of try-on effects including a first try-on effect and a second try-on effect, wherein the first try-on effect represents the effect of the first 3D avatar wearing the first garment and performing a first action, and the second try-on effect represents the effect of the first 3D avatar wearing the first garment and performing a second action, and the first try-on effect and the second try-on effect are different.

[0006] The aforementioned first simulation data can be obtained by the server through fabric simulation based on the first human body data, the first action data, and the first clothing data.

[0007] The first operation described above can be referred to in subsequent embodiments of this application. Figure 5B The operation of the virtual fitting control 512 is shown.

[0008] The above method, through edge-cloud collaboration, enables virtual try-on functionality on mobile devices, presenting high-precision dynamic virtual try-on effects. This allows users to virtually try on clothes while shopping in online apparel marketplaces, viewing a 3D avatar that matches their body type and how the clothing would look. These try-on effects realistically reflect how the clothing would appear on the user. This helps users understand how clothing will look on them before placing an order online, reducing the likelihood of buying unsuitable items and lowering return rates in the apparel e-commerce sector.

[0009] In conjunction with the first aspect, in some embodiments, the first human body data includes the first human body mesh data of the first 3D image, the first motion data includes the first skeleton data of the first motion and the second skeleton data of the second motion, and the first clothing data includes the clothing mesh data of the first clothing and clothing parameter information, the clothing parameter information including one or more of the following: the stretch coefficient, bending coefficient, and friction coefficient of the first clothing.

[0010] Alternatively, the first human body data includes human body parameters of the first 3D avatar, the human body parameters are used to generate the first human body mesh data, the first action data includes identification information of the first action and the second action, and the first clothing data includes identification information of the first clothing.

[0011] As can be seen from the above embodiments, electronic devices can reduce data transmission volume, save transmission bandwidth between electronic devices and servers, and improve the efficiency of virtual try-on by sending human body parameters, identification information of try-on actions (such as the first action and the second action), and identification information of the first garment to the server.

[0012] In conjunction with the first aspect, in some embodiments, the electronic device can determine that the first garment is suitable for virtual try-on; the electronic device provides a first control for selecting the first garment for virtual try-on; wherein the aforementioned first operation may be an operation on the first control.

[0013] In conjunction with the first aspect, in some embodiments, before the electronic device sends the first human body data of the first 3D avatar, the first action data of the first action sequence, and the first clothing data of the first garment to the server, the electronic device may also receive an operation to select the first 3D avatar and / or the first action sequence.

[0014] As shown in the above embodiments, users can choose to use their own or another person's 3D avatar for virtual try-on. During the virtual try-on process, users can adjust the clothing being tried on and the actions performed by the 3D avatar. The electronic device 100 can render the try-on effect of the 3D avatar wearing the clothing according to the user's selected clothing and actions, helping users accurately determine whether the clothing suits the user corresponding to the 3D avatar. This can effectively reduce the number of users buying unsuitable clothing online and lower the return rate of clothing e-commerce.

[0015] In conjunction with the first aspect, in some embodiments, the first set of fitting effects can be presented by playing a first video. The electronic device receives a second operation on the first video, the second operation being used to play the first video, or to pause playback of the first video, or to fast forward the first video, or to rewind the first video; the electronic device controls the playback of the first video according to the second operation.

[0016] During the playback of the first video, the first 3D avatar can wear the first outfit being tried on and perform the actions in the first action sequence. This allows the user to view a dynamic virtual try-on effect. Furthermore, the user can control the electronic device to pause the try-on effect, view it, and adjust the playback progress.

[0017] In conjunction with the first aspect, in some embodiments, the electronic device receives a third operation for selecting a second garment for virtual try-on; in response to the third operation, the electronic device sends second garment data and first layer information to the server, the first layer information indicating the layering of the first and second garments; the electronic device receives second simulation data from the server; the electronic device renders a second set of try-on effects based on the second simulation data, the second set of try-on effects including a third try-on effect and a fourth try-on effect, the third try-on effect representing the effect of the first 3D image wearing the first and second garments performing a first action, and the fourth try-on effect representing the effect of the first 3D image wearing the first and second garments performing a second action.

[0018] As can be seen from the above embodiments, users can choose to virtually try on a single garment or select multiple garments to view the effect of trying on multiple garment combinations.

[0019] In conjunction with the first aspect, in some embodiments, the electronic device receives a fourth operation, the fourth operation being used to select a third garment to replace the first garment for virtual try-on; in response to the fourth operation, the electronic device sends third garment data to the server; the electronic device receives third simulation data of the third garment from the server; the electronic device renders a third set of try-on effects based on the third simulation data, the third set of try-on effects including a fifth try-on effect and a sixth try-on effect, the fifth try-on effect being used to represent the effect of the first 3D image wearing the third garment performing a first action, and the sixth try-on effect being used to represent the effect of the first 3D image wearing the third garment performing a second action.

[0020] As can be seen from the above embodiments, during the virtual try-on process, users can select one or more garments to replace the garments already tried on by the 3D avatar. For example, users can adjust the size and color of the garments worn by the 3D avatar, or change the garments worn by the 3D avatar to other styles.

[0021] In conjunction with the first aspect, in some embodiments, the electronic device receives a fifth operation, which is used to transform a first action sequence into a second action sequence, the second action sequence including a third action and a fourth action; in response to the fifth operation, the electronic device sends second action data of the second action sequence to a server; the electronic device receives fourth simulation data of a first garment from the server; the electronic device renders a fourth set of fitting effects based on the fourth simulation data, the fourth set of fitting effects including a seventh fitting effect and an eighth fitting effect, the seventh fitting effect representing the effect of the first 3D image wearing the first garment performing the third action, and the eighth fitting effect representing the effect of the first 3D image wearing the first garment performing the fourth action.

[0022] In conjunction with the first aspect, in some embodiments, the electronic device receives a sixth operation, the sixth operation being used to select the fitting scene as the first scene; in response to the sixth operation, the electronic device acquires the first scene data of the first scene; the electronic device renders a first set of fitting effects based on the first scene data and the first simulation data, the first set of fitting effects including a first fitting effect and a second fitting effect, the first fitting effect being used to represent the effect of the first 3D image wearing the first clothing and performing a first action in the first scene, and the second fitting effect being used to represent the effect of the first 3D image wearing the first clothing and performing a second action in the first scene.

[0023] As shown in the above embodiments, during the virtual try-on process, users can set up try-on scenes, such as a runway, stage, square, grass, beach, indoors, etc. Electronic devices can render the try-on effect within these scenes. Providing one or more scene options within the virtual try-on experience increases its playability. Users can experience virtual try-on effects in different scenes.

[0024] In conjunction with the first aspect, in some embodiments, the electronic device acquires first body size information, which includes one or more of the following: height, shoulder width, chest circumference, waist circumference, hip circumference, arm circumference, arm length, neck circumference, abdominal circumference, hip circumference, thigh circumference, calf circumference, and leg length; the electronic device obtains first human mesh data of the first 3D image based on the first body size information.

[0025] Alternatively, the electronic device acquires one or more images, on which a human body corresponding to the first 3D image is displayed; the electronic device obtains first human body mesh data of the first 3D image based on one or more images.

[0026] Secondly, this application provides a virtual try-on method. This method can be executed by a processing module in an electronic device. The processing module can be a digital clothing simulation SDK. Specifically, the processing module can be used to: acquire first human body data of a first 3D avatar, first action data of a first action sequence, and first clothing data of a first garment, wherein the first action sequence includes a first action and a second action; send the first human body data, the first action data, and the first clothing data to a server; receive first simulation data of the first garment from the server; and render a first set of try-on effects using a 3D rendering module, the first set of try-on effects including a first try-on effect and a second try-on effect. The first try-on effect represents the effect of the first 3D avatar wearing the first garment and performing a first action, and the second try-on effect represents the effect of the first 3D avatar wearing the first garment and performing a second action. The first try-on effect and the second try-on effect are different.

[0027] The aforementioned first simulation data can be obtained by the server through fabric simulation based on the first human body data, the first action data, and the first clothing data.

[0028] The above method, through edge-cloud collaboration, enables virtual try-on functionality on mobile devices, presenting high-precision dynamic virtual try-on effects. This allows users to virtually try on clothes while shopping in online apparel marketplaces, viewing a 3D avatar that matches their body type and how the clothing would look. These try-on effects realistically reflect how the clothing would appear on the user. This helps users understand how clothing will look on them before placing an order online, reducing the likelihood of buying unsuitable items and lowering return rates in the apparel e-commerce sector.

[0029] In conjunction with the second aspect, in some embodiments, the first human body data includes the first human body mesh data of the first 3D image, the first motion data includes the first skeleton data of the first motion and the second skeleton data of the second motion, and the first clothing data includes the clothing mesh data of the first clothing and clothing parameter information, the clothing parameter information including one or more of the following: the stretch coefficient, bending coefficient, and friction coefficient of the first clothing.

[0030] Alternatively, the first human body data includes human body parameters of the first 3D avatar, the human body parameters are used to generate the first human body mesh data, the first action data includes identification information of the first action and the second action, and the first clothing data includes identification information of the first clothing.

[0031] As can be seen from the above embodiments, electronic devices can reduce data transmission volume, save transmission bandwidth between electronic devices and servers, and improve the efficiency of virtual try-on by sending human body parameters, identification information of try-on actions (such as the first action and the second action), and identification information of the first garment to the server.

[0032] In conjunction with the second aspect, in some embodiments, the processing module can be used to: preprocess the first simulation data to obtain preprocessed data, the preprocessing including one or more of the following: performing normal calculation on the first simulation data, synchronizing the first simulation data with the human body data of the first 3D image performing a first action sequence; and handing the preprocessed data to the 3D rendering module to render the first set of fitting effects.

[0033] The human body data of the first 3D avatar performing the first action sequence can include the human body mesh data corresponding to each action included in the first action sequence. The human body mesh data differs depending on the different actions performed by the 3D avatar.

[0034] In conjunction with the second aspect, in some embodiments, the processing module can be used to: acquire second garment data and first layer information of the second garment, the first layer information being used to indicate the layering of the first garment and the second garment; send the second garment data and the first layer information to the server; receive second simulation data from the server; and pass the second simulation data to the 3D rendering module to render a second set of fitting effects, the second set of fitting effects including a third fitting effect and a fourth fitting effect, the third fitting effect being used to represent the effect of the first 3D image wearing the first garment and the second garment performing a first action, and the fourth fitting effect being used to represent the effect of the first 3D image wearing the first garment and the second garment performing a second action.

[0035] As can be seen from the above embodiments, users can choose to virtually try on a single garment or select multiple garments to view the effect of trying on multiple garment combinations.

[0036] Thirdly, this application provides a virtual try-on method. This method can be applied to a communication system. The communication system may include an electronic device and a server. The electronic device receives a first operation, which is used to select a first garment for virtual try-on. In response to the first operation, the electronic device sends to the server first human body data of a first 3D avatar, first action data of a first action sequence, and first garment data of a first garment, wherein the first action sequence includes a first action and a second action. The server determines first simulation data of the first garment based on the first human body data, the first action data, and the first garment data, and sends the first simulation data to the electronic device. The electronic device renders a first set of try-on effects based on the first simulation data. The first set of try-on effects includes a first try-on effect and a second try-on effect. The first try-on effect represents the effect of the first 3D avatar wearing the first garment and performing a first action, and the second try-on effect represents the effect of the first 3D avatar wearing the first garment and performing a second action. The first try-on effect and the second try-on effect are different.

[0037] The above method, through edge-cloud collaboration, enables virtual try-on functionality on mobile devices, presenting high-precision dynamic virtual try-on effects. This allows users to virtually try on clothes while shopping in online apparel marketplaces, viewing a 3D avatar that matches their body type and how the clothing would look. These try-on effects realistically reflect how the clothing would appear on the user. This helps users understand how clothing will look on them before placing an order online, reducing the likelihood of buying unsuitable items and lowering return rates in the apparel e-commerce sector.

[0038] In conjunction with the third aspect, in some embodiments, the first human body data includes the first human body mesh data of the first 3D image, the first motion data includes the first skeleton data of the first motion and the second skeleton data of the second motion, and the first clothing data includes the clothing mesh data of the first clothing and clothing parameter information, the clothing parameter information including one or more of the following: the stretch coefficient, bending coefficient, and friction coefficient of the first clothing.

[0039] Alternatively, the first human body data includes human body parameters of the first 3D avatar, the human body parameters are used to generate the first human body mesh data, the first action data includes identification information of the first action and the second action, and the first clothing data includes identification information of the first clothing.

[0040] As can be seen from the above embodiments, electronic devices can reduce data transmission volume, save transmission bandwidth between electronic devices and servers, and improve the efficiency of virtual try-on by sending human body parameters, identification information of try-on actions (such as the first action and the second action), and identification information of the first garment to the server.

[0041] In conjunction with the third aspect, in some embodiments, the electronic device can determine that the first garment is suitable for virtual try-on; the electronic device provides a first control for selecting the first garment for virtual try-on; wherein the aforementioned first operation may be an operation on the first control.

[0042] In conjunction with the third aspect, in some embodiments, before the electronic device sends the first human body data of the first 3D avatar, the first action data of the first action sequence, and the first clothing data of the first clothing to the server, the electronic device may also receive an operation to select the first 3D avatar and / or the first action sequence.

[0043] As shown in the above embodiments, users can choose to use their own or another person's 3D avatar for virtual try-on. During the virtual try-on process, users can adjust the clothing being tried on and the actions performed by the 3D avatar. The electronic device 100 can render the try-on effect of the 3D avatar wearing the clothing according to the user's selected clothing and actions, helping users accurately determine whether the clothing suits the user corresponding to the 3D avatar. This can effectively reduce the number of users buying unsuitable clothing online and lower the return rate of clothing e-commerce.

[0044] In conjunction with the third aspect, in some embodiments, the first set of fitting effects can be presented by playing a first video. The electronic device receives a second operation on the first video, the second operation being used to play the first video, or to pause playback of the first video, or to fast forward the first video, or to rewind the first video; the electronic device controls the playback of the first video according to the second operation.

[0045] During the playback of the first video, the first 3D avatar can wear the first outfit being tried on and perform the actions in the first action sequence. This allows the user to view a dynamic virtual try-on effect. Furthermore, the user can control the electronic device to pause the try-on effect, view it, and adjust the playback progress.

[0046] In conjunction with the third aspect, in some embodiments, the electronic device receives a third operation for selecting a second garment for virtual try-on; in response to the third operation, the electronic device sends second garment data and first layer information to the server, the first layer information indicating the layering of the first and second garments; the server determines second simulation data by combining the second garment data and the first layer information, and sends the second simulation data to the electronic device; the electronic device renders a second set of try-on effects based on the second simulation data, the second set of try-on effects including a third try-on effect and a fourth try-on effect, the third try-on effect representing the effect of the first 3D image wearing the first and second garments performing a first action, and the fourth try-on effect representing the effect of the first 3D image wearing the first and second garments performing a second action.

[0047] As can be seen from the above embodiments, users can choose to virtually try on a single garment or select multiple garments to view the effect of trying on multiple garment combinations.

[0048] In conjunction with the third aspect, in some embodiments, the electronic device receives a fourth operation, which is used to select a third garment to replace the first garment for virtual try-on; in response to the fourth operation, the electronic device sends the third garment data to the server; the server determines the third simulation data of the third garment based on the first human body data, the first action data, and the third garment data, and sends the third simulation data to the electronic device; the electronic device renders a third set of try-on effects based on the third simulation data, the third set of try-on effects including a fifth try-on effect and a sixth try-on effect, the fifth try-on effect being used to represent the effect of the first 3D image wearing the third garment and performing a first action, and the sixth try-on effect being used to represent the effect of the first 3D image wearing the third garment and performing a second action.

[0049] As can be seen from the above embodiments, during the virtual try-on process, users can select one or more garments to replace the garments already tried on by the 3D avatar. For example, users can adjust the size and color of the garments worn by the 3D avatar, or change the garments worn by the 3D avatar to other styles.

[0050] In conjunction with the third aspect, in some embodiments, the electronic device receives a fifth operation, which is used to transform a first action sequence into a second action sequence, the second action sequence including a third action and a fourth action; in response to the fifth operation, the electronic device sends second action data of the second action sequence to the server; the server determines fourth simulation data of the first garment based on the first human body data, the second action data, and the first garment data, and sends the fourth simulation data to the electronic device; the electronic device renders a fourth set of fitting effects based on the fourth simulation data, the fourth set of fitting effects including a seventh fitting effect and an eighth fitting effect, the seventh fitting effect representing the effect of the first 3D image wearing the first garment and performing the third action, and the eighth fitting effect representing the effect of the first 3D image wearing the first garment and performing the fourth action.

[0051] In conjunction with the third aspect, in some embodiments, the electronic device receives a sixth operation, the sixth operation being used to select the fitting scene as the first scene; in response to the sixth operation, the electronic device acquires the first scene data of the first scene; the electronic device renders a first set of fitting effects based on the first scene data and the first simulation data, the first set of fitting effects including a first fitting effect and a second fitting effect, the first fitting effect being used to represent the effect of the first 3D image wearing the first clothing and performing a first action in the first scene, and the second fitting effect being used to represent the effect of the first 3D image wearing the first clothing and performing a second action in the first scene.

[0052] As shown in the above embodiments, during the virtual try-on process, users can set up try-on scenes, such as a runway, stage, square, grass, beach, indoors, etc. Electronic devices can render the try-on effect within these scenes. Providing one or more scene options within the virtual try-on experience increases its playability. Users can experience virtual try-on effects in different scenes.

[0053] In conjunction with the third aspect, in some embodiments, the electronic device acquires first body size information, which includes one or more of the following: height, shoulder width, chest circumference, waist circumference, hip circumference, arm circumference, arm length, neck circumference, abdominal circumference, hip circumference, thigh circumference, calf circumference, and leg length; the electronic device obtains first human mesh data of the first 3D image based on the first body size information.

[0054] Alternatively, the electronic device acquires one or more images, on which a human body corresponding to the first 3D image is displayed; the electronic device obtains first human body mesh data of the first 3D image based on one or more images.

[0055] In conjunction with the third aspect, in some embodiments, the server can obtain first human mesh data of the first 3D avatar based on first human body data, and obtain first skeletal data of the first action and second skeletal data of the second action based on first action data; the server uses the first human mesh data, first skeletal data, and second skeletal data to determine second human mesh data of the first 3D avatar under the first action and third human mesh data of the first 3D avatar under the second action; the server determines first simulation data based on the second human mesh data, third human mesh data, and first clothing data. The first simulation data may include simulation data of a collision between the first clothing and the first 3D avatar performing the first action, determined by the second human mesh data and first clothing data, and simulation data of a collision between the first clothing and the first 3D avatar performing the second action, determined by the third human mesh data and first clothing data.

[0056] In conjunction with the third aspect, in some embodiments, the server's database stores first simulation data; the server retrieves the first simulation data from the database based on first human body data, first action data, and first clothing data.

[0057] As can be seen from the above embodiments, the server can store the simulation data of clothing obtained from fabric simulation. When it is necessary to perform the same fitting action on the same or similar 3D image and put on the same clothing for fabric simulation again, the server can retrieve the clothing simulation data from the database without having to perform the specific calculation process of fabric simulation again. This can reduce the server's repetitive fabric simulation process, save server computing resources, and improve the server's concurrency capabilities.

[0058] Fourthly, this application provides an electronic device. The electronic device may include a communication device, a memory, and a processor. The communication device can be used to communicate between the electronic device and a server. The memory can be used to store a computer program. The processor can be used to invoke the computer program, causing the electronic device to execute any of the possible implementation methods described in the first aspect.

[0059] Fifthly, this application provides a communication system. The communication system may include an electronic device and a server. The electronic device may be used to receive a first operation, the first operation being used to select a first garment for virtual try-on; the electronic device may also be used, in response to the first operation, to send to the server first human body data of a first 3D avatar, first action data of a first action sequence, and first garment data of a first garment, wherein the first action sequence includes a first action and a second action; the server may be used to determine first simulation data of the first garment based on the first human body data, the first action data, and the first garment data, and send the first simulation data to the electronic device; the electronic device may be used to render a first set of try-on effects based on the first simulation data, the first set of try-on effects including a first try-on effect and a second try-on effect, the first try-on effect representing the effect of the first 3D avatar wearing the first garment and performing a first action, and the second try-on effect representing the effect of the first 3D avatar wearing the first garment and performing a second action, the first try-on effect and the second try-on effect being different.

[0060] In a sixth aspect, this application provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform any possible implementation method as described in the first aspect.

[0061] In a seventh aspect, this application provides a computer program product that may contain computer instructions that, when executed on an electronic device, cause the electronic device to perform any of the possible implementation methods in the first aspect.

[0062] Eighthly, this application provides a chip for use in an electronic device, the chip including one or more processors for invoking computer instructions to cause the electronic device to perform any of the possible implementation methods in the first aspect.

[0063] Understandably, the electronic device provided in the fourth aspect, the communication system provided in the fifth aspect, the computer-readable storage medium provided in the sixth aspect, the computer program product provided in the seventh aspect, and the chip provided in the eighth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0064] Figure 1 This is an architecture diagram of a communication system 10 provided in an embodiment of this application; Figure 2A This is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application; Figure 2B This is a software structure block diagram of an electronic device 100 provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a server 200 provided in an embodiment of this application; Figures 4A-4E These are some scene diagrams illustrating the creation of 3D images provided in the embodiments of this application; Figures 5A-5H These are schematic diagrams of some virtual try-on scenarios provided in the embodiments of this application; Figure 6 This is a flowchart of a virtual try-on method provided in an embodiment of this application; Figure 7 This is an architecture diagram of another communication system 70 provided in an embodiment of this application; Figure 8 This is an architecture diagram of another communication system 80 provided in an embodiment of this application; Figure 9 This is an architecture diagram of another communication system 90 provided in an embodiment of this application. Detailed Implementation

[0065] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0066] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0067] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0068] In the following embodiments of this application, the term "user interface (UI)" refers to the medium interface through which an application (APP) or operating system (OS) interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visual interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0069] In some embodiments, the electronic device can generate a clothing model based on clothing data and a human body model based on the user's human body data. Both the clothing model and the human body model can be 2D graphics. During virtual try-on, the electronic device can composite the clothing model as a texture with the human body model to render the effect of the user's virtual avatar wearing the clothing. However, since the clothing model is a static image, the try-on effect rendered by the electronic device cannot show the deformation caused by the collision between the clothing and the human body. That is, even if the user's virtual avatar performs different try-on actions, the try-on effect rendered by the electronic device using the aforementioned clothing model remains the same. The user cannot see the dynamic changes in the clothing as the virtual avatar performs different try-on actions.

[0070] This application provides a virtual try-on method. This method can be applied to a communication system including a mobile terminal and a server. The mobile terminal responds to a user's selection of clothing and a try-on action to perform a virtual try-on, sending the user's three-dimensional (3D) image data, 3D digital clothing data, and action data to the server. The try-on action can be a single action or a sequence of actions containing multiple actions. The server uses physical simulation technology to calculate the clothing deformation information after a collision between the human body and the clothing during the corresponding try-on action, obtaining simulation data of the digital clothing. The server can send the simulation data of the digital clothing to the mobile terminal. Then, the mobile terminal can perform real-time rendering based on the simulation data, displaying the try-on effect of the user's 3D image wearing the digital clothing. The try-on effect can realistically present the style, fabric, and other clothing information of the digital clothing, as well as the deformation information of details such as wrinkles caused by the collision between the digital clothing and the 3D image.

[0071] The aforementioned digital clothing can represent clothing obtained by modeling physical clothing.

[0072] The above method, through edge-cloud collaboration, enables virtual try-on functionality on mobile devices, presenting high-precision dynamic virtual try-on effects. This allows users to virtually try on clothes while shopping in online marketplaces, viewing a 3D avatar that matches their body type and shows how digital clothing would look on them. These try-on effects realistically reflect how the clothing would look on the user. This helps users understand how clothing will look on them before placing an order online, reducing the likelihood of buying unsuitable items and lowering return rates in the apparel e-commerce sector.

[0073] The concepts involved in the embodiments of this application are described below.

[0074] Mesh A mesh can refer to the grid of a 3D model. This 3D model could be, for example, a human body model, a clothing model, etc. Mesh data can include data representing the mesh of the 3D model. Mesh data can be used to render its corresponding mesh. A 3D model can be composed of polygons. A complex polygon can be composed of multiple triangular faces. Therefore, the surface of a 3D model can be composed of multiple interconnected triangular faces. In three-dimensional space, the set of points constituting these triangular faces and the edges of the triangles can be considered a mesh. For example, a human body mesh can include the points of the triangular faces constituting the surface of a human body model and the edges of the triangles. Human body mesh data can include the coordinates of the points constituting the surface of the human body model, the connection relationships between multiple points, etc. A clothing mesh can include the points of the triangular faces constituting the surface of a clothing model and the edges of the triangles. Clothing mesh data can include the coordinates of the points constituting the surface of a clothing model, the connection relationships between multiple points, etc.

[0075] In this model, the points that form the triangular faces of the 3D model are called vertices. Each vertex can correspond to one or more pieces of information, such as normals, UV texture coordinates, color, and tangents. This information can be used to render and paint the 3D model.

[0076] 3D models can be obtained through modeling. Modeling can refer to creating a mesh of a 3D model. A mesh can contain several main attributes (vertex coordinates, normals, texture coordinates, triangle drawing sequences, etc.). The process of creating a mesh can be the process of drawing triangles.

[0077] The architecture of the communication system involved in this application, as well as the hardware and software structures of the equipment, are described below.

[0078] Figure 1An exemplary architectural diagram of the communication system 10 provided in an embodiment of this application is shown.

[0079] like Figure 1 As shown, the communication system 10 may include an electronic device 100 and a server 200. A communication connection may be established between the electronic device 100 and the server 200. The electronic device 100 may be a mobile terminal. For example, the electronic device 100 may be a mobile phone, tablet computer, smartwatch, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. The specific type of the electronic device 100 is not limited in the embodiments of this application.

[0080] Electronic device 100 and server 200 can work together to provide users with a virtual try-on service. Users can select clothing they want to try on on electronic device 100 and view the effect of a 3D image matching their body shape wearing the clothing on electronic device 100.

[0081] like Figure 1 As shown, the electronic device 100 may include an e-commerce application 210. The e-commerce application 210 may be an app for providing online clothing purchase functionality. This application embodiment does not limit the name of the e-commerce application 210. The e-commerce application 210 may include an account management module 211, a product display module 212, an order placement module 213, a payment module 214, a 3D image management module 215, and a virtual try-on module 216.

[0082] The account management module 211 can be used to manage the account information of logged-in accounts in the e-commerce application 210. This account information may include, but is not limited to, account name, account password, etc. Based on the account management module 211, the e-commerce application 210 can provide users with functions such as account login, account cancellation, and modification of account name and password.

[0083] The product display module 212 provides product display functionality. These products may include clothing (such as tops, pants, skirts, shoes, etc.). Based on the product display module 212, the e-commerce application 210 can display product images, prices, parameters, and other product information. The product information displayed by the e-commerce application 210 can be uploaded by merchants registered with the e-commerce application 210.

[0084] The order placement module 213 can be used to provide the function of placing orders for goods. Based on the order placement module 213, users can place orders for goods in the e-commerce application 210.

[0085] The payment module 214 can be used to provide payment functionality. Based on the payment module 214, users can place an order in the e-commerce application 210 and then make payment.

[0086] This application embodiment does not limit the implementation method of providing account management function, product display function, order placement function and payment function of the above-mentioned e-commerce application 210.

[0087] The 3D avatar management module 215 provides 3D avatar creation functionality. Based on the 3D avatar management module 215, the e-commerce application 210 can create one or more 3D avatars. These 3D avatars can include the 3D avatar of the user corresponding to the logged-in account in the e-commerce application 210, and can also include the 3D avatars of other users. For example, user 1 can log in to their own account in the e-commerce application 210 and create their own 3D avatar so that they can use their 3D avatar for virtual try-on later. User 1 can also create 3D avatars of other users in the e-commerce application 210, such as the 3D avatar of user 1's mother. In this way, user 1 can also use their mother's 3D avatar for virtual try-on to buy clothes for their mother.

[0088] A 3D avatar can be obtained by creating a human body model of a user. A user's 3D avatar can realistically reflect that user's body shape. That is, the body dimensions of a user's 3D avatar in key body parts are the same as or similar to the user's actual body dimensions. Similarity in body dimensions means that the difference between the 3D avatar's body dimensions and the user's actual body dimensions is less than a preset threshold. The body dimensions may include one or more of the following: height, shoulder width, chest circumference, waist circumference, hip circumference, arm circumference, arm length, neck circumference, abdominal circumference, hip circumference, thigh circumference, calf circumference, leg length, etc. This application embodiment does not limit the content of the above body dimensions.

[0089] The method of creating 3D images by performing human body modeling in the 3D image management module 215 will be described in detail in subsequent embodiments, and will not be elaborated here.

[0090] In some embodiments, the 3D avatar management module 215 may be a module independent of the e-commerce application 210. For example, the 3D avatar management module 215 may be a system-level application provided by the electronic device 100. All e-commerce applications in the electronic device 100 can obtain the user's 3D avatar through the 3D avatar management module 215 and use the 3D avatar to provide virtual try-on functionality. In this way, the user can create a 3D avatar only once in the electronic device 100, without having to create 3D avatars multiple times in different e-commerce applications. This simplifies the user's 3D avatar creation process and improves the user experience.

[0091] The virtual try-on module 216 can be used to provide the function of displaying virtual try-on effects. The virtual try-on module 216 may include a 3D rendering module 216A, a digital clothing simulation software development kit (SDK) 216B, an action management module 216C, and a scene management module 216D.

[0092] The motion management module 216C can be used to manage the actions required for virtual try-on. In some embodiments, the actions can be a preset single action or a sequence of actions. In other embodiments, the actions can also be a sequence of actions captured in real time using motion capture technology. The action sequence is a set of multiple actions in a sequential order. Based on the motion management module 216C, the e-commerce application 210 can provide one or more action options for users to select try-on actions during virtual try-on. In the virtual try-on scenario, the 3D avatar can move according to the try-on actions selected by the user, thereby presenting the try-on effect of the 3D avatar wearing clothing and performing the try-on action.

[0093] Based on the action management module 216C, the e-commerce application 210 can also provide the function of recording virtual try-on actions. For example, the e-commerce application 210 can access the camera in the electronic device 100 to capture images containing user actions. Then, the e-commerce application 210 can analyze the user's actions from the images through the action management module 216C. These user actions can be used as virtual try-on actions.

[0094] The scene management module 216D can be used to manage the scenes required for virtual try-on. These scenes can represent the background when a 3D avatar performs a virtual try-on. For example, scenes can include a catwalk, stage, square, grass, beach, indoors, etc. This application embodiment does not limit the type of scene described above.

[0095] The Digital Clothing Simulation SDK216B can be used to interact with the server 200. The Digital Clothing Simulation SDK216B can be referred to as the processing module of the electronic device 100.

[0096] In some embodiments, the Digital Clothing Simulation SDK 216B can upload 3D image data, 3D digital clothing data, and motion data to the server 200.

[0097] The aforementioned 3D character data may include human mesh data. In some embodiments, the 3D character data may also include skeletal data. The human body model constructed from the 3D character data can represent the 3D character corresponding to the 3D character data.

[0098] The aforementioned 3D digital clothing data can include clothing mesh data and clothing parameter information. Clothing parameter information may include, but is not limited to, tensile coefficient, bending coefficient, and friction coefficient. Clothing parameter information can be determined based on the physical material of the fabric. 3D digital clothing data can be obtained by creating a clothing model.

[0099] The aforementioned motion data may include data corresponding to the fitting action. For example, the motion data may be the complete skeletal data of the fitting action. Optionally, the motion data may also be the motion ID corresponding to the fitting action.

[0100] The Digital Clothing Simulation SDK 216B can obtain the aforementioned motion data from the motion management module 216C. Then, the Digital Clothing Simulation SDK 216B can send the motion data to the server 200.

[0101] In some embodiments, the Digital Clothing Simulation SDK 216B can obtain simulation data of clothing obtained through physical simulation technology from the server 200. This clothing simulation data can be used to represent the shape of the clothing after a collision between the clothing and the human body during a fitting.

[0102] In one possible implementation, server 200 may include a 3D digital clothing simulation module 230. Specifically, the digital clothing simulation SDK 216B can interact with the 3D digital clothing simulation module 230.

[0103] In some embodiments, the Digital Clothing Simulation SDK 216B can also provide encoding and decoding functions to encode and decode data exchanged between the Digital Clothing Simulation SDK 216B and the server 200. For example, the Digital Clothing Simulation SDK 216B can encode data and send the encoded data to the 3D Digital Clothing Simulation module 230. The 3D Digital Clothing Simulation module 230 can then decode the received data. Encoding data by the Digital Clothing Simulation SDK 216B can reduce the amount of upstream data. The Digital Clothing Simulation SDK 216B can receive data sent from the 3D Digital Clothing Simulation module 230. This data may be encoded by the 3D Digital Clothing Simulation module 230. The Digital Clothing Simulation SDK 216B can decode the received data. The aforementioned encoding and decoding functions can reduce the transmission bandwidth between the Digital Clothing Simulation SDK 216B and the server 200.

[0104] In some embodiments, the Digital Clothing Simulation SDK 216B can also be used for preprocessing of clothing simulation data before rendering. For example, the Digital Clothing Simulation SDK 216B can perform normal calculations on the clothing simulation data.

[0105] The 3D rendering module 216A can be used to perform 3D rendering based on motion data, scene data, 3D image data, and clothing simulation data to display a virtual try-on effect. In one possible implementation, the image rendered by the 3D rendering module 216A can be a view from a fixed virtual camera perspective, or a view after switching the virtual camera perspective according to user control. The rendered image can also be zoomed in or out. Optionally, the rendered image can also be interactive based on user input. For example, the user input could be to apply a directional force to the clothing. The 3D rendering module 216A can then simulate a wind effect during 3D rendering based on this user input, presenting the effect of wind blowing the clothing in a specified direction.

[0106] Among them, the digital clothing simulation SDK 216B can send simulation data of clothing obtained from the server 200 to the 3D rendering module 216A. The motion management module 216C can send motion data to the 3D rendering module 216A. The scene management module 216D can send scene data to the 3D rendering module 216A.

[0107] In some embodiments, the electronic device 100 may further include a storage module ( Figure 1 (Not shown in the image). The electronic device 100 can store simulation data of clothing in the combination of 3D image + clothing + try-on action in the storage module. This simulation data can be sent by the server 200. In this way, when it is detected that the user selects the same 3D image + clothing + try-on action again for virtual try-on, the e-commerce application 210 can retrieve the simulation data of the clothing from the storage module without having to request the server 200 to perform fabric simulation again. The above embodiment can save the computing resources of the server 200 and the bandwidth of the electronic device 100.

[0108] In some embodiments, the providers of the digital clothing simulation SDK 216B and the e-commerce application 210 may be the same or different. The providers of the 3D image management module 215, the 3D rendering module 216A, the motion management module 216C, and the scene management module 216D may be the same or different.

[0109] Server 200 may include a 3D digital clothing management module 220 and a 3D digital clothing simulation module 230.

[0110] The 3D digital clothing simulation module 230 can be used to provide high-precision fabric simulation services. This fabric simulation can also be called clothing simulation. Fabric simulation involves calculating the deformation of the clothing mesh by analyzing the collision between the human body mesh and the clothing mesh. The 3D digital clothing simulation module 230 can determine the clothing simulation data based on 3D character data, 3D digital clothing data, and motion data. The clothing simulation data can include clothing mesh data. This clothing mesh data can be the mesh data after deformation caused by the collision between the clothing and the human body, reflecting the shape of the clothing and the human body after the collision during a fitting action.

[0111] The 3D digital clothing management module 220 can be used to manage 3D digital clothing data. Specifically, the 3D digital clothing management module 220 can store 3D digital clothing data obtained through clothing modeling (such as clothing mesh data and clothing parameter information).

[0112] In some embodiments, the e-commerce application 210 can communicate with the 3D digital clothing management module 220 to determine whether a garment has been modeled and has corresponding 3D digital clothing data. When it is determined that a garment has 3D digital clothing data, the e-commerce application 210 can provide operation controls for virtual try-on when displaying the garment. This allows the user to use these controls to virtually try on the garment. When it is determined that a garment does not have 3D digital clothing data, the e-commerce application 210 may not provide operation controls for virtual try-on when displaying the garment. That is, the user cannot virtually try on the garment.

[0113] In some embodiments, during the virtual try-on process, the digital clothing simulation SDK 216B in the e-commerce application 210 can send clothing identification information (such as clothing ID, etc.) to the 3D digital clothing simulation module 230. Then, the 3D digital clothing simulation module 230 can obtain the 3D digital clothing data of the clothing from the 3D digital clothing management module 220 based on the clothing identification information, thereby performing fabric simulation on the clothing. The amount of data for the clothing identification information is much smaller than the amount of data for clothing mesh data and clothing parameter information. It can be seen that the method of transmitting clothing identification information during the virtual try-on process can reduce the amount of data transmission between the end-user and the cloud, saving transmission bandwidth between the electronic device 100 and the server 200.

[0114] In some embodiments, the 3D digital clothing simulation module 230 can send the simulation data of the clothing obtained through fabric simulation to the 3D digital clothing management module 220. The 3D digital clothing management module 220 can store the simulation data of clothing in the combination of 3D image + clothing + fitting action. When it is necessary to perform the same fitting action and wear the same clothing for fabric simulation on the same or similar 3D image again, the 3D digital clothing simulation module 230 can obtain the simulation data of the clothing from the 3D digital clothing management module 220 without having to perform the specific calculation process of fabric simulation again. The above-mentioned similar 3D images can be represented by the difference in the body size information of the 3D images being less than a preset threshold. It can be seen that the above embodiments can reduce the server 200 from performing repeated fabric simulation processes, saving the server 200's computing resources.

[0115] In some embodiments, the electronic device 100 may include a 3D digital clothing simulation module. This allows the electronic device 100 to perform fabric simulation and render virtual fitting effects without interacting with the server 200.

[0116] In some embodiments, server 200 may include a 3D rendering module and a scene management module. In this way, server 200 can complete fabric simulation and rendering of virtual try-on effects. Server 200 can send the rendering results to electronic device 100. Electronic device 100 can display the virtual try-on effect based on the received rendering results.

[0117] The aforementioned electronic device 100 and server 200 may also include more or fewer modules. This application does not limit this.

[0118] The hardware and software structure of electronic device 100 is described here.

[0119] Figure 2A An exemplary schematic diagram of the hardware structure of electronic device 100 is shown.

[0120] like Figure 2AAs shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0121] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0122] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. In some embodiments, processor 110 may be a system-on-a-chip (SOC).

[0123] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0124] The processor 110 may also include a memory for storing instructions and data. In some examples, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or is recurring. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0125] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback.

[0126] The charging management module 140 receives charging input from a charger, which can be a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0127] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0128] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0129] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0130] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1.

[0131] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0132] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering.

[0133] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0134] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0135] The ISP is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye.

[0136] Camera 193 is used to capture still images or videos. In some embodiments, electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0137] In some embodiments, the e-commerce application in the electronic device 100 can call the camera 193 to capture images containing user actions. Then, the electronic device 100 can use motion capture technology to identify the user actions from the images and determine the user actions as fitting actions in the virtual fitting process.

[0138] In some embodiments, the electronic device 100 can also perform 3D rendering of the virtual try-on effect using a GPU.

[0139] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0140] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0141] In some embodiments, the electronic device 100 can perform human body modeling via an NPU to obtain a 3D image of the user. For example, the NPU can receive body size information of key body parts of the user and generate a 3D image of the user based on the aforementioned body size information. As another example, the NPU can receive one or more images containing a human body and generate a 3D image of the user based on the aforementioned images. This application embodiment does not limit the method of human body modeling using the NPU described above.

[0142] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0143] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0144] In some embodiments, the internal memory 121 may also be used to store one or more of the following data: 3D image data, clothing simulation data, fitting action data, fitting scene data, clothing identification information, etc.

[0145] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0146] Audio module 170 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. Audio module 170 can also be used for encoding and decoding audio signals. In some examples, audio module 170 may be located in processor 110, or some functional modules of audio module 170 may be located in processor 110. Speaker 170A, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. Receiver 170B, also called a "handset," is used to convert audio electrical signals into sound signals. Microphone 170C, also called a "microphone" or "microphone," is used to convert sound signals into electrical signals. Headphone jack 170D is used to connect wired headphones.

[0147] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, gravity sensors, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0148] The gyroscope sensor can be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor. The offset angle of the electronic device 100 can also be determined by the gyroscope sensor.

[0149] An accelerometer can detect the magnitude of acceleration of an electronic device 100 in various directions (typically three axes). In some embodiments, the accelerometer can be used to identify the posture of the electronic device 100 and can be applied to applications such as screen orientation switching and pedometers.

[0150] A gravity sensor can be used to determine the tilt angle of the electronic device 100 relative to a horizontal plane. In some embodiments, the screen state of the electronic device 100 can be determined by the gravity sensor, thereby adjusting the screen to keep it horizontal.

[0151] In some embodiments, the electronic device 100 can determine the distance traveled by the electronic device 100 over a period of time using an acceleration sensor and a gravity sensor.

[0152] Buttons 190 include a power button, volume buttons, etc. Motor 191 can generate vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, and also to indicate messages, missed calls, notifications, etc.

[0153] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and detach from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The electronic device 100 interacts with the network through the SIM card to achieve functions such as calls and data communication. In some examples, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be removed from it.

[0154] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android® system as an example to illustrate the software structure of electronic device 100.

[0155] Figure 2B This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0156] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android® system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android Runtime and system libraries, and the kernel layer.

[0157] The application layer can include a series of application packages.

[0158] like Figure 2B As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, SMS, and e-commerce applications. Among them, the e-commerce application can provide functions such as online clothing purchase.

[0159] The application framework layer provides APIs and a programming framework for applications within the application layer. The application framework layer includes some predefined functions.

[0160] like Figure 2B As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, activity manager, etc.

[0161] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0162] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0163] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0164] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection, hang-up, etc.).

[0165] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0166] The notification manager allows applications to display notifications in the status bar (such as the pull-down notification bar). It can be used to convey informational messages and can disappear automatically after a short pause without user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0167] The Activity Manager is responsible for managing activities, including launching, switching, and scheduling components within the system, as well as managing and scheduling applications. The Activity Manager can be invoked by upper-level applications to open the corresponding activities.

[0168] The Android Runtime consists of core libraries and a virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

[0169] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0170] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0171] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0172] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0173] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0174] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing. Among them, the electronic device 100 can use the 3D graphics processing library to realize 3D rendering of virtual try-on, and display the user's 3D image on the screen to show the effect of trying on clothes.

[0175] A 2D graphics engine is a graphics engine for 2D drawing.

[0176] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0177] Figure 3 An exemplary schematic diagram of a server 200 provided in an embodiment of this application is shown.

[0178] like Figure 3 As shown, server 200 may include one or more processors 310, memory 311, communication interface 312, transmitter 314, receiver 315, coupler 316, and antenna 317. These components may be connected via bus 313 or other means. Figure 3 The following explanation uses a bus connection as an example. Specifically: Communication interface 312 can be used by server 200 and other electronic devices, such as Figure 1 The electronic device 100 shown communicates with the device. Specifically, the communication interface 312 can be a 3G communication interface, a Long Term Evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, etc. Not limited to wireless communication interfaces, the server 200 can also be configured with a wired communication interface 312 to support wired communication.

[0179] In some embodiments of this application, transmitter 314 and receiver 315 can be considered as a wireless modem. Transmitter 314 can be used to transmit signals output by processor 310. Receiver 315 can be used to receive signals. In server 200, the number of transmitters 314 and receivers 315 can be one or more. Antenna 317 can be used to convert electromagnetic energy in a transmission line into electromagnetic waves in free space, or to convert electromagnetic waves in free space into electromagnetic energy in a transmission line. Coupler 316 can be used to split mobile communication signals into multiple paths and distribute them to multiple receivers 315. Understandably, antenna 317 of server 200 can be implemented as a large-scale antenna array.

[0180] The memory 311 is coupled to the processor 310 and is used to store various software programs and / or multiple sets of instructions. Specifically, the memory 311 may include high-speed random access memory and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

[0181] The memory 311 may store an operating system (hereinafter referred to as the system), such as an embedded operating system like uCOS®, VxWorks®, or RTLinux®. The memory 311 may also store a network communication program that can be used to communicate with one or more electronic devices (such as electronic device 100).

[0182] In this embodiment, the memory 311 can be used to store clothing data, clothing simulation data, and motion data of fitting actions. The clothing data may include one or more of the following: clothing identification information (such as clothing ID), and 3D digital clothing data. The 3D digital clothing data may include clothing mesh data and clothing parameter information. The motion data may include one or more of the following: motion identification information (such as motion ID), and motion skeletal data.

[0183] Optionally, the memory 311 can also be used to store the base model data of the human body base model consistent with the electronic device 100 side, so that the server 200 can generate 3D image data of the user consistent with the electronic device 100 side based on the base model data.

[0184] In this embodiment, the processor 310 can be used to read and execute computer-readable instructions. Specifically, the processor 310 can be used to call a program stored in the memory 311 and execute the instructions contained in the program. This program can, for example, be an implementation program for fabric simulation provided in one or more embodiments of this application.

[0185] It should be noted that, Figure 3 The server 200 shown is merely one implementation of the embodiments of this application. In actual applications, the server 200 may include more or fewer components, which is not limited here.

[0186] based on Figure 1 The communication system 10 shown below will be used to describe some virtual fitting scenarios provided in the embodiments of this application.

[0187] Figures 4A-4E Examples of scene diagrams for creating 3D images are shown.

[0188] like Figure 4A As shown, the electronic device 100 can display a user interface 410. The user interface 410 may include application icons. For example, an e-commerce application icon 411. In response to an operation on the e-commerce application icon 411, the electronic device 100 can run the e-commerce application and display... Figure 4B The user interface shown is 420.

[0189] like Figure 4BAs shown, the user interface 420 can be a product display interface in an e-commerce application. The user interface 420 may include product options 421, 422, 423, 424, etc. Each product option may display corresponding product information (such as product images, text descriptions, prices, etc.). For example, product option 421 could be a product option for pants. Product option 421 may include an image of the pants, a text description of the pants, and the price of the pants. Product option 421 also includes a virtual try-on control 421A. The virtual try-on control 421A can be used to virtually try on the clothing corresponding to product option 421.

[0190] Product option 422 can be a product option corresponding to a shirt. Product option 422 may contain a virtual try-on control 422A. The virtual try-on control 422A can be used to virtually try on the clothing corresponding to product option 422.

[0191] Product option 423 can be a product option corresponding to a skirt. Product option 423 may also include virtual try-on controls.

[0192] Product option 424 can be the product option corresponding to a short-sleeved shirt. Product option 424 does not include virtual try-on controls. This means that the clothing corresponding to product option 424 cannot be virtually tried on.

[0193] In one possible implementation, the e-commerce application in the electronic device 100 can determine whether to provide the aforementioned virtual try-on control in its corresponding product options based on whether a garment can be virtually tried on. Specifically, the e-commerce application can determine whether the garment can be virtually tried on based on whether the server 200 stores 3D digital garment data. If the server 200 stores 3D digital garment data, then the garment can be virtually tried on. Otherwise, the garment cannot be virtually tried on.

[0194] The user interface 420 may also include my control 425. In response to an operation on my control 425, the electronic device 100 can display... Figure 4C The user interface shown is 430.

[0195] like Figure 4C As shown, the user interface 430 may include a user information display area 431, a 3D avatar control 432, and other controls.

[0196] User information display area 431 can be used to display account information of logged-in accounts in e-commerce applications. For example, account avatar, account name, etc.

[0197] The 3D avatar control 432 can be used to trigger the electronic device 100 to open the management interface of the 3D avatar. In response to the operation of the 3D avatar control 432, the electronic device 100 can display... Figure 4D The user interface shown is 440.

[0198] like Figure 4D As shown, the user interface 440 may include newly created controls 441 and existing 3D image areas 442.

[0199] The newly created control 441 can be used to create new 3D images.

[0200] The existing 3D avatar area 442 can display options corresponding to the 3D avatars that users (e.g., user 1) have already created in the e-commerce application. For example, 3D avatar options 442A and 442B. 3D avatar option 442A can correspond to user 1's own 3D avatar. 3D avatar option 442B can correspond to user 1's mother's 3D avatar.

[0201] In response to Figure 4D The operation shown for the newly created control 441 can be displayed on the electronic device 100. Figure 4E The user interface shown is 450.

[0202] like Figure 4E As shown, the user interface 450 may include an information input area 451, a confirm control 452, and a cancel control 453.

[0203] The information input area 451 can be used to input information for creating a 3D avatar. This information may include one or more body dimensions, such as height, weight, shoulder width, chest circumference, waist circumference, hip circumference, arm circumference, arm length, etc. This application embodiment does not limit the information used for creating a 3D avatar.

[0204] The control 452 can be used to trigger the electronic device 100 to create a 3D image based on the information entered in the information input area 451.

[0205] Cancel control 453 can be used to cancel the creation of a 3D avatar.

[0206] Not limited to use Figure 4E In addition to using one or more body size information to create a 3D image, the electronic device 100 can also use other methods for human body modeling to create a 3D image. For example, the electronic device 100 can also use one or more images containing a human body to create a 3D image. Therefore, in response to... Figure 4D As shown in the operation of creating a new control 441, the electronic device 100 can provide an image upload entry to prompt the user to upload one or more images containing human bodies. Then, the electronic device 100 can perform human body modeling based on the one or more images containing human bodies uploaded by the user to create a 3D image.

[0207] Electronic device 100 creates a 3D avatar and can define and save the 3D avatar data. This 3D avatar data may include human mesh data and skeletal data. This 3D avatar data can be used for virtual try-on, allowing users to view the effect of clothing on the 3D avatar.

[0208] The entry point for creating 3D avatars can be provided not only by e-commerce applications but also by system-level applications within the electronic device 100. That is, users can create 3D avatars not only within individual e-commerce applications but also within system-level applications that provide the entry point for 3D avatar creation. These system-level applications can provide an interface for obtaining 3D avatars. Each e-commerce application within the electronic device 100 can obtain 3D avatars from the aforementioned system-level applications through this interface.

[0209] From the above Figures 4A-4E As shown in the scenario, electronic device 100 can provide the function of creating 3D avatars, allowing users to create 3D avatars that match their own or other users' body types. In this way, during virtual try-on, users can see the effect of the 3D avatar wearing the clothing. This try-on effect can realistically reflect how the clothing looks on the user corresponding to the 3D avatar, helping users more accurately understand whether the clothing suits them or other users.

[0210] Figures 5A-5H Examples of virtual try-on scenarios are shown below.

[0211] like Figure 5A As shown, the electronic device 100 can display a user interface 420. The user interface 420 may include product options 422. The product options 422 may include a virtual try-on control 422A. Other controls in the user interface 420 can be found in the description of the foregoing embodiments.

[0212] In response to Figure 5A The operation of product option 422 shown is displayed on the electronic device 100. Figure 5B The user interface 510 shown is a display interface for the clothing corresponding to product option 422.

[0213] like Figure 5B As shown, the user interface 510 may include an image display area 511, a virtual try-on control 512, a shopping cart control 513, and a purchase control 514, etc.

[0214] Image display area 511 can be used to display images of clothing.

[0215] The virtual fitting control 512 can be used to display clothing (i.e., clothing) on ​​the user interface 510. Figure 5AVirtual try-on is available for the clothing item corresponding to product option 422 shown.

[0216] The shopping cart control 513 can be used to add clothing displayed on the user interface 510 to the shopping cart.

[0217] Purchase control 514 can be used to purchase clothing displayed in user interface 510.

[0218] In some embodiments, the electronic device 100 stores multiple 3D images. For example, as described above. Figure 4D The 3D images corresponding to 3D image options 442A and 442B are shown. In response to... Figure 5B The virtual try-on control shown is 512 or Figure 5A The operation of the virtual fitting control 422A shown can be displayed on the electronic device 100. Figure 5C The user interface 520 is shown. User interface 520 may include multiple options corresponding to 3D avatars in electronic device 100, allowing the user to select the 3D avatar for virtual try-on. For example, user interface 520 may include 3D avatar option 521 and 3D avatar option 522. 3D avatar option 521 may be the option corresponding to the user's own 3D avatar. 3D avatar option 522 may be the option corresponding to the user's mother's 3D avatar. Understandably, if a user needs to buy clothing for themselves, they can select their own 3D avatar for virtual try-on. If a user needs to buy clothing for their mother, they can select their mother's 3D avatar for virtual try-on.

[0219] The user interface 520 may also include a confirm control 523 and a cancel control 524. The confirm control 523 can be used to trigger the electronic device 100 to perform a virtual try-on based on the 3D image corresponding to the selected 3D image option in the user interface 520. The cancel control 524 can be used to cancel the virtual try-on.

[0220] In some embodiments, the electronic device 100 stores only one 3D image. In response to... Figure 5B The virtual try-on control shown is 512 or Figure 5A The virtual try-on control 422A shown allows the electronic device 100 to directly use the 3D image stored in the electronic device 100 for virtual try-on without displaying the actual device. Figure 5C The user interface shown is simply for users to select a 3D avatar.

[0221] In some embodiments, the electronic device 100 does not store a 3D avatar. That is, the user has not yet created a 3D avatar. In response to Figure 5B The virtual try-on control shown is 512 or Figure 5AThe operation of the virtual try-on control 422A shown can prompt the user to create a 3D avatar. For example, the electronic device 100 can display the aforementioned... Figure 4E The user interface 450 shown in the image instructs the user to create a 3D avatar. After the user creates the 3D avatar, the electronic device 100 can use the user-created 3D avatar for virtual try-on.

[0222] For example, in Figure 5C When the 3D image option 521 is selected, the electronic device 100 can display [the desired image] in response to the operation of the confirmation control 523. Figure 5D The user interface shown is 530.

[0223] like Figure 5D As shown, the user interface 530 may include a fitting effect display area 531, a pause control 532, a playback progress bar 533, a recommended clothing option area 534, an action option area 535, and a scene option area 536.

[0224] The fitting room effect display area 531 can be used to display the fitting room scene and the fitting room effect of a 3D avatar wearing the clothing and performing the fitting room action. The aforementioned fitting room effect can be obtained by the electronic device 100 through 3D rendering based on the 3D avatar data, action data, and clothing simulation data. The aforementioned clothing simulation data can be obtained by the electronic device 100 from the server 200.

[0225] The pause control 532 can be used to pause the playback of the try-on effect.

[0226] The playback progress bar 533 can be used to indicate the current playback progress of the virtual try-on effect.

[0227] It should be noted that, in some embodiments, the above-mentioned fitting effect can be presented in the form of video playback. The fitting action can be a sequence of actions with a sequential order. During the video playback presenting the fitting effect, the 3D avatar can wear the clothing being tried on and perform the actions in the sequence in sequence.

[0228] In this way, users can view the dynamic virtual try-on effect. Furthermore, users can use the pause control 532 to pause the playback of the try-on effect on the electronic device 100. Users can also view and adjust the playback progress of the try-on effect using the playback progress bar 533.

[0229] In other embodiments, the fitting effect displayed in the fitting effect display area 531 may also be static. For example, the fitting action may be a single action. The electronic device 100 may display a 3D rendered image of a 3D figure wearing the clothing being tried on and performing a single fitting action in the fitting effect display area 531.

[0230] The recommended clothing options area 534 may contain one or more clothing options. The electronic device 100 can match the clothing corresponding to the selected clothing option in the recommended clothing options area 534 with the aforementioned... Figure 5B The user interface 510 displays clothing combinations for virtual try-on. For example, the recommended clothing option area 534 may include shirt options, skirt options, and clothing option 1. The clothing option corresponding to the shirt option can be one of the aforementioned options. Figure 5B The user interface 510 shows a shirt. The shirt and skirt options are selected. The electronic device 100 can virtually try on shirts by matching the shirt option with the skirt option. It can be seen that... Figure 5D In the virtual fitting effect display area 531 shown, the 3D avatar wears the shirt and skirt selected by the user for virtual try-on, demonstrating the upper body effect of the shirt and skirt to the user.

[0231] In some embodiments, when virtual try-on is performed with multiple garments, the electronic device 100 can determine the layering information of these garments. This layering information can represent the overlapping levels of the garments in a 3D image. For example, a shirt can be layered over a skirt, partially obscuring the skirt. Alternatively, a skirt can be layered over a shirt, partially obscuring the shirt. The layering levels of these garments can be specified by the user. The electronic device 100 can provide a layering level specification control in the user interface 530, allowing the user to specify the layering levels of the garments. This application embodiment does not limit the user operation for specifying layering levels described above.

[0232] Optionally, when virtual try-on is performed with multiple garments, the electronic device 100 can also determine the position and wearing status of one or more garments during the try-on process. For example, if the garments being tried on include a skirt, the electronic device 100 can determine the height of the skirt on the 3D image (e.g., high-waisted, mid-waisted, or low-waisted, etc.). As another example, if the garments being tried on include a coat, the electronic device 100 can determine the wearing status of the coat (e.g., whether the coat buttons are fastened or unfastened). This application embodiment does not limit the specific content of the above-mentioned wearing information. The aforementioned position and wearing status information can be specified by the user. This application embodiment does not limit the user operations used to specify the wearing information.

[0233] Understandably, users are not limited to selecting the above-mentioned skirt option in the recommended clothing option area 534. They can also select more clothing options in the recommended clothing option area 534 to match the shirts displayed in the user interface 510 for virtual try-on.

[0234] In other words, users can choose to virtually try on a single piece of clothing, or they can choose to mix and match multiple pieces of clothing to see how the outfits look.

[0235] Optionally, the electronic device 100 can also select the clothing corresponding to the selected clothing option in the recommended clothing option area 534. Figure 5B The user interface 510 displays clothing replacements for virtual try-on. For example, if the selected clothing option in the recommended clothing option area 534 includes a top, this top can replace the shirt displayed in the user interface 510. The try-on effect display area 531 can show the 3D avatar wearing the replacement shirt. In some embodiments, the electronic device 100 can provide replacement controls in the user interface 530. These replacement controls can be used to replace the clothing currently being tried on by the 3D avatar with one or more garments in a virtual try-on scenario. This application embodiment does not limit the display style of the replacement controls.

[0236] In other words, during the virtual try-on process, users can choose one or more garments to replace the ones already worn by the 3D avatar. For example, users can adjust the size and color of the garment worn by the 3D avatar, or change the garment to a different style.

[0237] In some embodiments, the electronic device 100 can be configured to allow the user to initially select clothing for trying on (as described above). Figure 5B The user interface 510 shows a shirt as an example to determine recommended clothing, and displays clothing options corresponding to the recommended clothing in the recommended clothing option area 534. For example, the recommended clothing may include clothing that can be matched with the clothing initially selected by the user for trying on. This application embodiment does not limit the implementation method of the electronic device 100 determining the recommended clothing.

[0238] The action option area 535 may contain one or more action options, which can be used by the user to select the desired try-on action. For example, the action option area 535 may contain action 1, action 2, and action 3 options. The try-on action corresponding to an action option can be a single action or a sequence of multiple actions in a specific order. These try-on actions can be preset. Optionally, these try-on actions can be captured in real time using motion capture technology.

[0239] In one possible implementation, the electronic device 100 can provide a function for recording fitting actions, allowing users to record their fitting movements. Specifically, the electronic device 100 uses a camera to capture user movements through motion capture technology. The electronic device 100 can store the captured user movement data and provide corresponding action options in the action option area 535. Thus, users can select the action option corresponding to their recorded fitting actions in the action option area 535, thereby viewing the fitting effect presented by the 3D avatar wearing the clothing and performing their recorded fitting actions.

[0240] In other words, users can record their desired or preferred fitting actions and choose to use their recorded actions when trying on clothes virtually.

[0241] In another possible implementation, the electronic device 100 can provide a real-time fitting action matching function. This function can be used to match the actions of the 3D avatar with the user's actions in real time during virtual fitting. That is, the 3D avatar can wear the clothing being tried on and mimic the user's actions in real time. For example, if the user walks forward, the 3D avatar can also walk forward. If the user spins in a circle, the 3D avatar can also spin in a circle. If the user raises their arm, the 3D avatar can also raise their arm. During the virtual fitting process, the electronic device 100 can keep its camera on, using motion capture technology to capture the user's actions in real time and identify these actions as fitting actions. The electronic device 100 can then render a 3D avatar wearing the clothing and mimicking the user's actions in real time.

[0242] In other words, during the virtual try-on process, users can pose in various ways to see the effect of the 3D avatar wearing the clothes and performing the corresponding actions.

[0243] The scene selection area 536 may contain one or more scene options, which can be used by the user to select a scene for virtual try-on. For example, the scene selection area 536 may contain scene 1, scene 2, and scene 3 options. This application embodiment does not limit the above-mentioned scenes.

[0244] This explanation uses the dynamic fitting effect displayed in the fitting effect display area 531 as an example.

[0245] contrast Figure 5D and Figure 5E It can be seen that, Figure 5E The fitting effect display area 531 shows the 3D image performing fitting actions and... Figure 5D The 3D figures in the display area 531 shown are performing different fitting actions. Figure 5E The playback progress bar shown is 533 times. Figure 5D The playback progress bar shown is 533 units long. Figure 5EThe playback progress of the virtual try-on effect shown is longer than... Figure 5D The playback progress of the virtual try-on effect is shown. It can be seen that under different try-on actions, the 3D image and the clothing will collide differently, resulting in different deformations of the clothing. The electronic device 100 can perform 3D rendering based on the simulation data of the clothing under different try-on actions, presenting the try-on effect of the 3D image wearing the clothing under different try-on actions. Users can view the deformation of the clothing when the 3D image performs different try-on actions. For example, the try-on actions that the 3D image needs to perform may include a first action and a second action. The electronic device 100 can perform 3D rendering based on the simulation data of a garment under the first action to obtain the first try-on effect, and perform 3D rendering based on the simulation data of the same garment under the second action to obtain the second try-on effect. The first and second try-on effects can be used to represent the effect of the same 3D image wearing the same garment under the first and second actions, respectively. Because the first and second actions are different, the deformation of the clothing caused by the collision between the 3D image and the clothing under different actions is also different. Therefore, the first and second try-on effects are different.

[0246] contrast Figure 5F and Figure 5E It can be seen that, Figure 5F The fitting effect display area 531 shows the 3D image performing fitting actions and... Figure 5E The 3D figures in the display area 531 shown are performing different fitting actions. Figure 5F The playback progress bar shown is 533 times. Figure 5E The playback progress bar shown is 533 units long. Figure 5F The playback progress of the virtual try-on effect shown is longer than... Figure 5E The playback progress of the virtual try-on effect is shown. Similarly, the electronic device 100 can display the virtual try-on effect of a 3D image wearing clothing and performing different try-on actions. Users can see the deformation of the clothing as the 3D image performs different try-on actions.

[0247] contrast Figure 5G and Figure 5F It can be seen that, Figure 5G The fitting effect display area 531 shows the 3D image performing fitting actions and... Figure 5F The 3D figures in the display area 531 shown are performing different fitting actions. Figure 5G The playback progress bar shown is 533 times. Figure 5F The playback progress bar shown is 533 units long. Figure 5G The playback progress of the virtual try-on effect shown is longer than... Figure 5FThe playback progress of the virtual try-on effect is shown. Similarly, the electronic device 100 can display the virtual try-on effect of a 3D image wearing clothing and performing different try-on actions. Users can see the deformation of the clothing as the 3D image performs different try-on actions.

[0248] Understandably, the above Figures 5D to 5G The content displayed in the fitting effect display area 531 can be a partial fitting effect diagram of a 3D figure wearing clothes and performing a continuous fitting action in the same scene.

[0249] like Figure 5G As shown, the scene option area 536 may include scene option 536A. In response to an operation on scene option 536A, the electronic device 100 can switch the virtual try-on scene from the scene corresponding to scene option 1 to the scene corresponding to scene option 536A.

[0250] As shown in Figure 5H, the electronic device 100 can display, in the fitting effect display area 531, a 3D avatar wearing clothing and performing a fitting action within the scene corresponding to scene option 536A of scene 2. The selected scene option in scene option area 536 is determined by... Figure 5G The options for Scenario 1 shown have changed to Figure 5H The scenario shown is option 536A.

[0251] It can be seen that, Figures 5D to 5G The scene presented in the fitting effect display area 531 shown is similar to... Figure 5H The scenes presented in the fitting effect display area 531 are different.

[0252] Understandably, since the 3D image, the clothing being tried on, and the fitting process remain unchanged, Figure 5H The 3D image and the clothing being tried on, displayed in the fitting effect display area 531, can be compared with... Figures 5D to 5G The 3D image and the clothing being tried on are displayed consistently in the fitting effect display area 531. When only the scene is switched, the electronic device 100 can play the same 3D image wearing the same clothing and performing the same fitting action again in a new scene (such as the scene corresponding to scene 2 option 536A).

[0253] In some embodiments, the electronic device 100 may store preset human body models. Users can perform virtual try-on using existing human body models in the electronic device 100 without creating a 3D avatar. For example, the electronic device 100 may provide male and female human body models. Users can select either a male or female human body model to try on one or more garments. The electronic device 100 can render the try-on effect of the selected human body model wearing the selected garment based on the garment selected by the user and the human body model. Optionally, users can also select try-on actions and / or try-on scenes to view the try-on effect of the preset male or female human body model wearing the selected garment and performing specified try-on actions in a specified try-on scene. The server 200 may store human body models consistent with those on the electronic device 100. The electronic device 100 can send the human body model's identification information (such as human body model ID) to the server 200 based on the human body model selected by the user, and send the garment's identification information (such as garment ID) to the server 200 based on the garment selected by the user. Server 200 can perform fabric simulation based on the identification information of the human body model and the clothing to obtain clothing simulation data. Server 200 can then send the clothing simulation data to electronic device 100. Electronic device 100 can then render the fitting effect based on the received simulation data. Optionally, electronic device 100 can also send the 3D image data of the human body model selected by the user to server 200. The above method of using a preset human body model for virtual clothing fitting can help users refer to the effect of the clothing on their body, so that they can determine whether the clothing suits them.

[0254] From the above Figures 5A-5H As shown in the scenario, users can virtually try on clothes using their own or someone else's 3D avatar. During the virtual try-on process, users can adjust the clothing, the 3D avatar's actions, and the virtual environment. The electronic device 100 can render the effect of the 3D avatar wearing the clothing and performing the actions in the selected environment, helping users accurately determine whether the clothing suits the user corresponding to the 3D avatar. This can effectively reduce the number of users buying unsuitable clothing online and lower the return rate for clothing e-commerce.

[0255] Figure 6 A flowchart of a virtual try-on method provided in an embodiment of this application is shown as an example.

[0256] like Figure 6 As shown, this method can be applied to a communication system that includes an electronic device 100 and a server 200. (Refer to the foregoing...) Figure 1The electronic device 100 may include a product display module 212, a 3D image management module 215, a 3D rendering module 216A, a digital clothing simulation SDK 216B, an action management module 216C, and a scene management module 216D. The server 200 may include a 3D digital clothing management module 220 and a 3D digital clothing simulation module 230. This virtual try-on method may include steps S611~S622. Wherein: 1. (S611~S615) Determine the clothing to be tried on, the trying-on action, and the trying-on scene.

[0257] S611, the product display module 212 can send a virtual try-on request to the 3D rendering module 216A.

[0258] In some embodiments, the product display module 212 can detect user actions involving virtual try-on of one or more garments. These user actions can be referred to the foregoing. Figure 5B The operation of the virtual try-on control 512 is shown. Then, the product display module 212 can send a virtual try-on request to the 3D rendering module 216A. This virtual try-on request can contain description information about the clothing. This description information can be used by the 3D rendering module 216A to determine which one or more garments are to be tried on.

[0259] Not limited to the product display module 212, the aforementioned virtual try-on request can also be sent by other modules. For example, the electronic device 100 may include a virtual try-on detection module. This module can detect the clothing selected by the user for virtual try-on. Then, the virtual try-on detection module can send the aforementioned virtual try-on request. The module that sends the virtual try-on request can be configured by the e-commerce application as needed.

[0260] Not limited to the aforementioned 3D rendering module 216A, the aforementioned virtual try-on request can also be received by other modules.

[0261] This application uses the example of the product display module 212 sending a virtual try-on request to the 3D rendering module 216A for illustration.

[0262] In some embodiments, in addition to the description information of the clothing, the virtual try-on request may also include the description information of the target user. The target user can be a user who needs to virtually try on clothing. For example, as... Figure 5C As shown, users can select a 3D avatar for which they wish to virtually try on clothing. The product display module 212 can determine the target user based on the selected 3D avatar option in the user interface 520. The 3D avatar corresponding to the selected 3D avatar option in the user interface 520 is the 3D avatar of the target user. This target user information can be used by the 3D rendering module 216 to determine the target user for virtual try-on.

[0263] S612, 3D rendering module 216A can obtain human body data from 3D image management module 215.

[0264] Based on the target user's target information in the aforementioned virtual try-on request, the 3D rendering module 216 can obtain the target user's human body data.

[0265] In one possible implementation, the aforementioned human body data can be 3D avatar data of the target user. The 3D avatar data can include human mesh data and skeletal data. The body size information of the aforementioned human mesh data can be the same as or similar to the target user's actual body size information. For example, the height, weight, shoulder width, chest circumference, waist circumference, hip circumference, and arm length of the human model constructed from the human mesh data can be the same as or similar to the target user's actual height, weight, shoulder width, chest circumference, waist circumference, hip circumference, and arm length, respectively. The human body can be composed of multiple skeletons, such as the torso, limbs, and head. The aforementioned skeletal data can be used to represent the topological structure of the target user's skeleton and can describe one or more movements of the target user's entire body. The target user's skeletal data can be used to correspond with the fitting action to determine the target user's 3D avatar's human mesh data under the fitting action.

[0266] In some embodiments, the 3D avatar data may further include facial mesh information. This facial mesh information can be used to give the 3D avatar more realistic and natural expressions when displaying virtual try-on effects. The facial mesh information can be obtained by modeling the target user's face.

[0267] In some embodiments, the aforementioned 3D image data may further include hair mesh information. This hair mesh information can be used to represent the effect of the target user's hair during a virtual try-on process. This allows the user to understand whether the clothing matches their hairstyle. The hair mesh information can be obtained by modeling the target user's hair.

[0268] In some embodiments, the 3D avatar data may further include shoe mesh information. This shoe mesh information can be used to represent the virtual try-on effect of the 3D avatar wearing different shoes. This allows users to see if the clothing matches their shoes. The shoe mesh information can be obtained by modeling the target user's shoes.

[0269] The aforementioned 3D image data may also include more or less content, such as mesh information for earrings, necklaces, handbags, sunglasses, etc. This application does not limit this aspect.

[0270] In another possible implementation, the aforementioned human body data can be based on a set of human body schema weight information. The aforementioned human body schema can be a preset human body model, which may correspond to schema data. The schema data may include schema mesh data and schema skeleton data. The server 200 may store schema data consistent with that of the electronic device 100. Thus, the server 200 can generate human body mesh data consistent with that of the electronic device 100 based on the schema mesh data and the schema weight information of the target user, and generate skeletal data consistent with that of the electronic device 100 based on the schema skeleton data and the schema weight information of the target user.

[0271] In addition to the aforementioned fundamental model weight information, the human body data can also be other parameters that can be used to generate human body mesh data for the target user. For example, parameters of a skinned multi-person linear model (SMPL). SMPL represents a parameterized human body model that can use a set of parameters (such as body shape parameters, posture parameters, etc.) to express the mesh information of the human body.

[0272] The amount of data in the aforementioned fundamental model weight information or other parameters that can be used to generate the target user's human body mesh data is usually less than the amount of data in the aforementioned 3D image data. Therefore, by acquiring the target user's fundamental model weight information or other parameters that can be used to generate the target user's human body mesh data, the 3D rendering module 216A can reduce the amount of data transmitted in subsequent steps S616 and S617, save the transmission bandwidth between the electronic device 100 and the server 200, and improve the efficiency of virtual try-on.

[0273] The parameters used to generate the target user's human mesh data can be referred to as human body parameters. As can be seen from the foregoing embodiments, human body parameters can be, for example, fundamental model weight information, or SMPL parameters, or body size information (such as one or more of the following: height, shoulder width, chest circumference, waist circumference, hip circumference, arm circumference, arm length, neck circumference, abdominal circumference, hip circumference, thigh circumference, calf circumference, leg length, etc.). This application embodiment does not limit the specific content of the aforementioned human body parameters.

[0274] In some embodiments, the 3D image data of the target user can be obtained through human body modeling and already stored in the 3D image management module 215. The 3D image management module 215 can acquire the user's body size information and generate 3D image data such as human body mesh data and skeletal data based on the body size information. Alternatively, the 3D image management module 215 can generate the user's 3D image data through traditional manual modeling or scan modeling. For example, the 3D image management module 215 can acquire one or more images containing the user's human body. The 3D image management module 215 can use image recognition methods (such as human keypoint recognition) to process the one or more images to generate 3D image data such as human body mesh data and skeletal data. The above-described method of generating 3D image data through human body modeling is merely an exemplary illustration of this application. The 3D image management module 215 can also use other methods to construct the human body model.

[0275] Optionally, the 3D image management module 215 can use the 3D image data and base model data obtained through human body modeling to generate the aforementioned base model weight information.

[0276] The S613 and 3D rendering module 216A can obtain clothing data from the 3D digital clothing management module 220.

[0277] Based on the description information of the clothing in the virtual try-on request, the 3D rendering module 216A can obtain the clothing data of the clothing.

[0278] In one possible implementation, the clothing data acquired by the 3D rendering module 216A may include 3D digital clothing data of the clothing being tried on. This 3D digital clothing data may include clothing mesh data and clothing parameter information.

[0279] The garment mesh data can include multiple garment sub-mesh data. For example, garment sub-mesh data can include material mesh data, part mesh data, etc. Material mesh data can be used to indicate the material of the garment (such as silk, cotton, linen, leather, etc.). Part mesh data can be used to indicate the mesh data of a specific part of the garment (such as sleeves, collar, etc.). Alternatively, garment sub-mesh data can include mesh data combining the garment's material and part.

[0280] Clothing parameter information may include one or more of the following: tensile coefficient, bending coefficient, coefficient of friction, etc. Clothing parameter information can be determined based on the physical material of the clothing fabric. In some embodiments, clothing parameter information may also include connection information between different clothing sub-mesh data, such as bending direction information at the connection points of different parts of the clothing.

[0281] In another possible implementation, the clothing data acquired by the 3D rendering module 216A can be identification information of the clothing being tried on, such as a clothing ID. This clothing identification information can be used by the server 200 to determine the 3D digital clothing data corresponding to the clothing ID. The amount of data for the clothing identification information is much smaller than the amount of data for the 3D digital clothing data. Therefore, transmitting the clothing identification information between the electronic device 100 and the server 200 can reduce the amount of data transmitted and save transmission bandwidth between the electronic device 100 and the server 200.

[0282] Among them, the same garment with different sizes (such as S, M, L, etc.) can have different garment IDs, or can be distinguished by a combination of garment ID and size code. This can more realistically simulate the fitting effect of different sizes of clothing on the same 3D avatar, so that users can choose the appropriate size of clothing.

[0283] The 3D digital clothing data for the same garment can be different even if the sizes are different.

[0284] In some embodiments, the garment being tried on can be a single garment or a combination of multiple garments. In the case of virtually trying on multiple garments, the 3D rendering module 216A can also acquire the layer information of these multiple garments. This layer information can represent the overlapping layers of these garments in the 3D image. That is, the garment data can also include layer information of the garments. For example, in... Figure 5D In the virtual try-on scenario of a shirt and skirt, the shirt and skirt are layered differently. The shirt can be layered over the skirt, partially obscuring it. Similarly, in the virtual try-on scenario of a shirt and jacket, the jacket and skirt are layered differently. The jacket can be layered over the shirt, partially obscuring it. This application does not limit the layering information of multiple garments in its embodiments.

[0285] The layering information for the aforementioned multiple garments can be preset or specified by the user. For example, a user can specify that the shirt is layered over the skirt, so that the shirt partially covers the skirt. Alternatively, a user can specify that the skirt is layered over the shirt, so that the skirt partially covers the shirt.

[0286] In some embodiments, after obtaining clothing data from the 3D digital clothing management module 220, the 3D rendering module 216A can store the clothing data. When it is necessary to obtain the clothing data of the same garment again, the 3D rendering module 216A can read the clothing data from the memory of the electronic device 100. In this way, for the same garment, the 3D rendering module 216A does not need to request the 3D digital clothing management module 220 to send clothing data multiple times. This can reduce the amount of data transmitted between the electronic device 100 and the server 200 and improve the efficiency of virtual try-on.

[0287] In some embodiments, the aforementioned 3D digital clothing data may be obtained through clothing modeling and already stored on server 200, for example, stored in the 3D digital clothing management module 220 of server 200. The aforementioned 3D digital clothing data can be created using professional digital clothing modeling software. Alternatively, 3D digital clothing data can also be obtained by adding clothing parameter information to a mesh model output by a digital content creation (DCC) tool. The aforementioned DCC tool can be a 3D modeling tool, such as Maya, 3ds Max, Blender, etc. This application embodiment does not limit the method for creating clothing models and generating 3D digital clothing data described above.

[0288] In some embodiments, step S613 is optional. The electronic device 100 and the server 200 may store consistent identification information (such as a garment ID) for the same garment. The 3D rendering module 216A can determine the garment's identification information based on the garment's description information in the virtual try-on request, and then send the garment's identification information to the 3D digital garment simulation module 230 via the digital garment simulation SDK 216B.

[0289] S614, the 3D rendering module 216A can obtain motion data from the motion management module 216C.

[0290] The motion data acquired by the 3D rendering module 216A can be the motion data of the 3D image to perform the fitting actions during virtual fitting.

[0291] In one possible implementation, motion data may include skeletal data. This skeletal data can describe one or more human movements. Specifically, the skeletal data for a fitting room movement can represent the topological structure of the skeleton when the human body performs this movement. The number of bones determines the level of detail in representing the fitting room movement. The number of bones in the skeletal data representing a fitting room movement can range from a dozen to hundreds. The number of bones can be determined according to the needs of the virtual fitting room service. This application does not limit the number of bones in the skeletal data for a fitting room movement.

[0292] Different fitting actions can correspond to different skeletal data. For example, the first action can correspond to the first skeletal data, and the second action can correspond to the second skeletal data. When the fitting actions required for the 3D character include multiple actions, the action data acquired by the 3D rendering module 216A can include the skeletal data corresponding to multiple actions. For example, the fitting actions required for the 3D character include the first action and the second action. The action data acquired by the 3D rendering module 216A can include the first skeletal data and the second skeletal data.

[0293] In another possible implementation, the motion data may include identification information of the fitting action, such as an action ID. The server 200 may store identification information of fitting actions consistent with those on the electronic device 100 side. In this way, the server 200 can determine the fitting action based on the identification information, and then perform fabric simulation based on the collision between the 3D image and the clothing during that fitting action to obtain clothing simulation data.

[0294] The amount of data for the identification information of the aforementioned fitting action is much smaller than the amount of data for the skeletal data. Therefore, the 3D rendering module 216A can reduce the amount of data transmission in subsequent steps S616 and S617 by acquiring the identification information of the fitting action, thereby saving transmission bandwidth between the electronic device 100 and the server 200 and improving the efficiency of virtual fitting.

[0295] The aforementioned motion data can be preset fitting motion data, user-selected fitting motion data, or user motion data collected in real time using motion capture technology. This application embodiment does not limit the specific content of the aforementioned fitting motion.

[0296] The S615 and 3D rendering module 216A can obtain scene data from the scene management module 216D.

[0297] The scene data mentioned above may include scene mesh data. The scene mesh data of a scene can be obtained by modeling that scene. The scene for virtual try-on can be, for example, a runway, a stage, a square, a lawn, a beach, a room, etc.

[0298] In some embodiments, the scene data may also include lighting information. This lighting information can be used by the 3D rendering module 216A to simulate the effects of sunny days, cloudy days, or different combinations of stage lighting during rendering.

[0299] In some embodiments, the scene data may also include particle information. This particle information can be used by the 3D rendering module 216A to simulate effects such as rain or snow during rendering.

[0300] This application does not limit the content of the scene data in its embodiments. The scene data may also contain more or less content.

[0301] The scene management module 216D can store scene data for one or more scenes, allowing users to select a fitting scene during virtual try-on. Providing one or more scene options within the virtual try-on scene increases its playability. Users can experience the virtual try-on effects in different scenes.

[0302] 2. (S616~S618) Perform fabric simulation on the clothing.

[0303] The S616 and 3D rendering module 216A can send human body data, motion data, and clothing data to the digital clothing simulation SDK 216B to request clothing simulation data.

[0304] S617, the Digital Clothing Simulation SDK216B can send digital clothing simulation requests to the 3D Digital Clothing Simulation Module 230.

[0305] The Digital Clothing Simulation SDK216B can be used to handle information exchange between electronic device 100 and server 200, and can encode and decode data, and send and receive data.

[0306] When the 3D rendering module 216 receives a request to obtain simulation data of clothing, the digital clothing simulation SDK 216B can send a digital clothing simulation request to the 3D digital clothing simulation module 230. This digital clothing simulation request may include the human body data, motion data, and clothing data from step S616 above. The digital clothing simulation SDK 216B can compress and encode the aforementioned human body data, motion data, and clothing data before sending them to the 3D digital clothing simulation module 230. This can reduce the amount of data transmission between the electronic device 100 and the server 200, saving transmission bandwidth.

[0307] In some embodiments, the Digital Clothing Simulation SDK216B can also provide functions such as identity authentication and authorization.

[0308] The S618 and 3D digital clothing simulation module 230 can perform digital clothing simulation and obtain simulation data.

[0309] In one possible implementation, the 3D digital clothing simulation module 230 can calculate the human mesh data of the target user under one or more actions based on the human body data and motion data in the digital clothing simulation request, using a skeletal skinning algorithm. These one or more actions are the fitting actions that the target user's 3D avatar needs to perform during virtual fitting.

[0310] The 3D digital clothing simulation module 230 can reuse the same fitting action for the target user based on the target user's skeletal data and the skeletal data of a fitting action, thereby obtaining the target user's skeletal data under that fitting action. This action redirection allows the same fitting action to be reused for different target users and prevents the 3D images of different target users from losing proportions or undergoing unnecessary deformation when performing the same fitting action.

[0311] Then, the 3D digital clothing simulation module 230 can generate human mesh deformation data of the target user based on the target user's human mesh data and skeletal data during a fitting action using a skeletal skinning algorithm, thereby determining the human mesh data of the target user during this fitting action. In other words, the deformation of the target user's human model under the skeletal data driven by the fitting action can be determined by the skeletal skinning algorithm.

[0312] Furthermore, the 3D digital clothing simulation module 230 can determine the clothing simulation data after the target user's 3D avatar collides with the clothing during one or more actions, based on the target user's human mesh data under one or more actions and the clothing data in the digital clothing simulation request. The aforementioned clothing simulation data may include clothing mesh data. This clothing mesh data can be the mesh data resulting from the deformation caused by the collision between the clothing and the target user's 3D avatar, reflecting the shape of the clothing after the collision (which may include deformation details such as wrinkles on the clothing).

[0313] In some embodiments, the motion redirection based on the target user's skeletal data and the skeletal data of the fitting action is optional. The skeletal data of the fitting action can be used as the target user's skeletal data to drive the target user's human mesh. Specifically, the 3D digital clothing simulation module 230 can obtain the target user's 3D image data based on the received human body data. This 3D image data may include the target user's human mesh data. The 3D digital clothing simulation module 230 can use a skeletal skinning algorithm to generate the target user's human mesh deformation data based on the target user's human mesh data and the skeletal data of the fitting action, thereby determining the target user's human mesh data during this fitting action. That is to say, the 3D image data may also not include the user's skeletal data.

[0314] The above embodiments can reduce the computational load during the creation of 3D avatars and fabric simulation. Furthermore, when the differences between the arm length, leg length, and height information corresponding to the skeletal data of the fitting action and the actual arm length, leg length, and height of the target user are small, the above embodiments can more accurately obtain the human mesh data of the target user performing the fitting action, and the rendered virtual fitting effect can more realistically reflect the effect of the target user wearing the clothing.

[0315] Understandably, when a 3D avatar needs to perform multiple fitting actions, the aforementioned clothing simulation data can include clothing mesh sequences. A clothing mesh sequence can contain multiple clothing mesh data points. One clothing mesh data point corresponds to one fitting action, reflecting the shape of the clothing after a collision between the clothing and the target user's 3D avatar during a fitting action.

[0316] It should be noted that if the human body data in the digital clothing simulation request is basic model weight information, the 3D digital clothing simulation module 230 can obtain the basic model data from the 3D digital clothing management module 220, and determine the target user's human body mesh data based on the basic model mesh data and basic model weight information in the basic model data, and determine the target user's skeletal data based on the basic model skeleton data and basic model weight information in the basic model data. Alternatively, the 3D digital clothing simulation module 230 can send the basic model weight information to the 3D digital clothing management module 220. The 3D digital clothing management module 220 can store basic model data. The 3D digital clothing management module 220 can determine the target user's 3D image data based on the basic model data and basic model weight information, and send the target user's 3D image data to the 3D digital clothing simulation module 230. In this way, the 3D digital clothing simulation module 230 can perform fabric simulation based on the target user's 3D image data to obtain the clothing simulation data.

[0317] If the clothing data in the digital clothing simulation request is the clothing identification information, the 3D digital clothing simulation module 230 can obtain the 3D digital clothing data of the clothing from the 3D digital clothing management module 220 according to the clothing identification information, so as to use the 3D digital clothing data for fabric simulation.

[0318] If the motion data in the digital clothing simulation request is the identification information of the fitting action, the 3D digital clothing simulation module 230 can obtain the skeletal data of the fitting action from the 3D digital clothing management module 220 according to the identification information of the fitting action, so as to use the skeletal data of the fitting action to perform fabric simulation.

[0319] 3. (S619~S622) Render the fitting effect.

[0320] The S619 and 3D digital clothing simulation module 230 can send simulation data to the digital clothing simulation SDK 216B.

[0321] The S620 and Digital Clothing Simulation SDK216B can preprocess simulation data.

[0322] The aforementioned preprocessing may include calculating the Normal data corresponding to the clothing mesh data in the simulation data to optimize the rendering effect of the 3D rendering module 216A in subsequent step S622. This application embodiment does not limit the specific processing procedure of the aforementioned preprocessing.

[0323] S621, the Digital Clothing Simulation SDK216B can send pre-processed simulation data to the 3D rendering module 216A.

[0324] In one possible implementation, step S620 is optional. That is, the preprocessing described above does not necessarily have to be performed by the Digital Clothing Simulation SDK216B.

[0325] For example, the preprocessing described above can be performed by the 3D rendering module 216A. Specifically, the digital clothing simulation SDK 216B can send the received simulation data to the 3D rendering module 216A. After preprocessing the simulation data, the 3D rendering module 216A then executes the following step S622.

[0326] For example, the aforementioned preprocessing can be performed by the 3D digital clothing simulation module 230. After obtaining clothing simulation data through fabric simulation, the 3D digital clothing simulation module 230 can perform the aforementioned preprocessing on the simulation data. Then, the 3D digital clothing simulation module 230 can send the preprocessed simulation data to the digital clothing simulation SDK 216B. The digital clothing simulation SDK 216B then sends the preprocessed simulation data to the 3D rendering module 216A.

[0327] The S622 and 3D rendering module 216A can complete rendering based on 3D character data, motion data, scene data, and pre-processed simulation data.

[0328] In some embodiments, during the virtual try-on process, the target user's 3D avatar needs to perform multiple try-on actions. The human mesh data of the target user's 3D avatar is different when performing different try-on actions, and the simulation data of the clothing is also different. Each try-on action can correspond to human mesh data and clothing simulation data. After receiving the preprocessed simulation data, the 3D rendering module 216A can synchronize the human mesh data and the preprocessed simulation data to ensure that the 3D avatar can wear the clothing constructed from the simulation data corresponding to that try-on action, reducing the problem of clothing clipping.

[0329] Optionally, the aforementioned data synchronization can also be performed by the Digital Clothing Simulation SDK 216B. That is, the preprocessing of simulation data by the Digital Clothing Simulation SDK 216B may also include data synchronization between the human mesh data and the simulation data. The Digital Clothing Simulation SDK 216B can send the data synchronization information to the 3D rendering module 216A. When rendering, the 3D rendering module 216A can combine and render the human mesh data and clothing simulation data corresponding to a fitting action based on the data synchronization information. The data obtained after preprocessing the simulation data by the Digital Clothing Simulation SDK 216B can be called preprocessed data. As can be seen from the foregoing embodiments, the preprocessed data may include one or more of the following: Normal data obtained by performing Normal calculations on the clothing simulation data, and information obtained by synchronizing the human mesh data and simulation data corresponding to the fitting action (i.e., data synchronization information). This data synchronization information can be used to reduce clothing clipping problems when the 3D rendering module 216A renders the fitting effect. This application embodiment does not limit the content of the aforementioned preprocessed data. The preprocessed data may also include more or less content. The Digital Clothing Simulation SDK 216B can send preprocessed data to the 3D rendering module 216A.

[0330] Optionally, the aforementioned data synchronization can also be performed by the 3D digital clothing simulation module 230. The 3D digital clothing simulation module 230 can send synchronized simulation data and human body mesh data to the digital clothing simulation SDK 216B. The synchronized simulation data and human body mesh data can represent simulation data and human body mesh data corresponding to the same fitting action. The digital clothing simulation SDK 216B can then send the synchronized simulation data and human body mesh data to the 3D rendering module 216A.

[0331] The 3D rendering module 216A can call the rendering pipeline to render the fitting scene based on scene data, and render the fitting effect of the 3D image wearing clothes and performing one or more fitting actions based on 3D image data, motion data and preprocessed simulation data.

[0332] In some embodiments, the 3D rendering module 216A can acquire preset lighting information and virtual camera viewpoint information, and render the fitting effect under preset lighting and virtual camera viewpoint conditions. Optionally, the 3D rendering module 216A can also receive user interaction information to adjust the rendered content of the fitting effect.

[0333] For example, the aforementioned user interaction information could be determined based on the user's adjustment of the virtual camera's viewing angle. The 3D rendering module 216A can render the virtual try-on effect after the virtual camera's viewing angle has been adjusted. This allows users to manually adjust the viewing angle to see the virtual try-on effect from different perspectives.

[0334] For example, the aforementioned user interaction information could be determined based on the user's actions to adjust the lighting. The 3D rendering module 216A can then render the fitting room effect after the lighting adjustment.

[0335] For example, the aforementioned user interaction information could be information determined based on the operation of adjusting the fitting action. The 3D rendering module 216A can obtain simulation data of the clothing after the fitting action adjustment from the 3D digital clothing simulation module 230 through the digital clothing simulation SDK 216B, and then render the fitting effect of the 3D image wearing the clothing after the adjustment. The method by which the 3D rendering module 216A obtains the simulation data of the clothing after the fitting action adjustment can be referred to the description of steps S616~S621 above.

[0336] The embodiments of this application do not limit the specific content of the above-mentioned user interaction information.

[0337] Depend on Figure 6 It is evident that the virtual try-on method provided in this application can achieve high-precision virtual try-on effects on mobile terminals with limited computing power. Furthermore, the cloud-side fabric simulation + terminal-side rendering mode can reduce the computing cost of server 200 and improve its concurrency capabilities. Server 200 can simultaneously respond to fabric simulation requests from multiple mobile terminals, collaborating with these terminals to provide virtual try-on services to multiple users. Thus, when users purchase clothing online using e-commerce applications on their mobile terminals, they can conveniently view the try-on effect of a 3D avatar matching their body type, allowing them to determine if the clothing suits them. This effectively reduces the likelihood of users buying unsuitable clothing online, lowering the return rate for clothing e-commerce. Moreover, since both the user's human body model and the clothing model can be completed before virtual try-on, electronic device 100 and server 200 can directly use the established human body and clothing models when collaboratively providing virtual try-on services. This effectively improves the efficiency of virtual try-on, allowing electronic device 100 to quickly present the virtual try-on effect to the user.

[0338] Another communication system 70 provided in the embodiments of this application is described below.

[0339] Figure 7 An exemplary architecture diagram of a communication system 70 is shown. Figure 7 As shown, the communication system 70 may include an electronic device 710 and a server 720.

[0340] The electronic device 710 may include an action management module 711, a 3D image management module 712, a digital clothing simulation SDK 713, a 3D rendering module 714, and a scene management module 715. The action management module 711, 3D image management module 712, digital clothing simulation SDK 713, 3D rendering module 714, and scene management module 715 can be referenced respectively as described above. Figure 1 Introduction to the Action Management Module 216C, 3D Image Management Module 215, Digital Clothing Simulation SDK 216B, 3D Rendering Module 216A, and Scene Management Module 216D shown.

[0341] The scene management module 715 is optional. In some embodiments, the electronic device 710 may also omit the scene management module 715.

[0342] The Digital Clothing Simulation SDK 713 may include an encoding / decoding module 713A, a network transmission module 713B, and a rendering preprocessing module 713C. The encoding / decoding module 713A is used to encode data to be sent and to decode received data. The network transmission module 713B is used for communication between the Digital Clothing Simulation SDK 713 and the 3D Digital Clothing Simulation module 724 in the server 720. The rendering preprocessing module 713C is used to preprocess the received clothing simulation data. For example, it performs normal calculations on the clothing simulation data.

[0343] Among them, the Digital Clothing Simulation SDK713 can be referred to as the processing module of the electronic device 710.

[0344] Server 720 may include a 3D digital clothing management module 721, an action management module 722, a 3D image management module 723, and a 3D digital clothing simulation module 724. The 3D digital clothing management module 721 and the 3D digital clothing simulation module 724 can respectively refer to the aforementioned... Figure 1 The following is an introduction to the 3D digital clothing management module 220 and the 3D digital clothing simulation module 230 shown.

[0345] The 3D digital clothing simulation module 724 may include a fabric simulation module 724A, an encoding / decoding module 724B, and a network transmission module 724C. The fabric simulation module 724A can be used to perform fabric simulation based on 3D image data, skeletal data of the fitting action, and 3D digital clothing data to obtain clothing simulation data. The encoding / decoding module 724B and the network transmission module 724C can be referred to in the aforementioned descriptions of the encoding / decoding module 713A and network transmission module 713B in the electronic device 710, respectively.

[0346] The motion management module 722 can be used to determine the fitting action based on the fitting action ID or other motion identifier information, thereby obtaining the skeletal data of the fitting action. In one possible implementation, the motion management module 722 may store the fitting action ID consistent with that on the electronic device 710, as well as the skeletal data of the fitting action. The motion management module 722 can look up the skeletal data of the corresponding fitting action based on the fitting action ID.

[0347] The 3D image management module 723 can be used to generate human body mesh data consistent with that of the electronic device 710 based on human body mesh parameters.

[0348] In one possible implementation, the aforementioned human mesh parameters can be based on a set of human baseline template weight information. The 3D avatar management module 723 can store baseline template data consistent with that of the electronic device 710. The 3D avatar management module 723 generates 3D avatar data of the target user consistent with that of the electronic device 710 based on the baseline template data and the target user's baseline template weight information.

[0349] In another possible implementation, the aforementioned human mesh parameters may include the user's body size information (such as shoulder width, chest circumference, waist circumference, hip circumference, leg length, arm length, etc., one or more of which). The 3D image management module 723 may store a human mesh generation model consistent with the side of the electronic device 710. The 3D image management module 723 can generate human mesh data consistent with the side of the electronic device 710 based on the human mesh parameters and the mesh generation model.

[0350] In another possible implementation, the aforementioned human mesh parameters can be other parameters that can be used to generate human mesh data. For example, SMPL parameters, etc. This application does not limit the specific content of the aforementioned human mesh parameters.

[0351] like Figure 7 As shown, the digital clothing simulation SDK 713 in the electronic device 710 can obtain human mesh parameters from the 3D image management module 712 and the fitting action ID from the action management module 711. The digital clothing simulation SDK 713 can also obtain the clothing ID of the garment to be tried on. The digital clothing simulation SDK 713 can send the human mesh parameters, fitting action ID, and clothing ID to the 3D digital clothing simulation module 724 in the server 720. The human mesh parameters and fitting action ID are described above. The clothing ID can be used to identify the garment. When there are multiple garments to be tried on, the clothing ID sent by the digital clothing simulation SDK 713 can be the clothing IDs of multiple garments.

[0352] In some embodiments, the garment ID of a garment can be a combination of the garment's style ID and its size information (such as size S, M, L, etc.). The style ID identifies the style of the garment. It is understood that garments of the same style can have different sizes. Garments of the same style but different sizes typically look different on a user. This allows users to adjust the size during virtual try-on to find the most suitable garment for themselves.

[0353] In some embodiments, the garment ID of a garment can be a combination of the garment's manufacturer ID, style ID, and garment size information.

[0354] In other embodiments, where the garment ID includes the garment IDs of multiple garments, the garment ID may also include hierarchical information of these multiple garments.

[0355] The embodiments of this application do not limit the specific content of the above-mentioned clothing ID.

[0356] The 3D digital clothing simulation module 724 can use the received human body mesh parameters to instruct the 3D image management module 723 to determine the human body mesh data.

[0357] The 3D digital clothing simulation module 724 can obtain the skeletal data of the fitting action from the action management module 722 using the received fitting action ID. The fitting action can be a single action or a sequence of actions containing multiple actions.

[0358] The 3D digital clothing simulation module 724 can use the received clothing ID to obtain the 3D digital clothing data (such as clothing mesh, clothing parameter information, etc.) of the clothing from the 3D digital clothing management module 721.

[0359] The 3D digital clothing simulation module 724 can utilize the acquired human mesh data, skeletal data of the fitting action, and 3D digital clothing data to calculate and reconstruct the human mesh data corresponding to each fitting action. Furthermore, the 3D digital clothing simulation module 724 can perform fabric simulation, calculating simulation data after a collision occurs between the clothing and the 3D figure during the fitting action. The clothing simulation data can describe the deformation effects such as bending and wrinkling of the clothing when the 3D figure wears it during the fitting action. The fabric simulation calculation can be real-time (e.g., 30 frames per second), non-real-time (e.g., 1 frame per second), or ultra-real-time (e.g., 100 frames per second). This application embodiment does not limit the calculation rate. The specific process of the fabric simulation can be referred to the aforementioned... Figure 6 The following is an introduction to steps S616~S618.

[0360] The 3D digital clothing simulation module 724 can send the clothing simulation data to the digital clothing simulation SDK 713 in the electronic device 710.

[0361] The digital clothing simulation SDK 713 can preprocess the simulation data of clothing and send the preprocessed simulation data to the 3D rendering module 714. The preprocessing can include calculating the normal information of each clothing mesh frame in the simulation data. The preprocessing can also include synchronizing the clothing simulation data with the human mesh of the 3D avatar during a fitting action. The simulation data and human mesh data corresponding to the same fitting action can be a single combination. In this way, the 3D rendering module 714 can render based on the synchronized data, ensuring that the 3D avatar can wear the clothing constructed from the simulation data corresponding to that fitting action when performing a fitting action, reducing clothing clipping issues.

[0362] Optionally, the digital clothing simulation SDK 713 can also send the received simulation data to the 3D rendering module 714. The 3D rendering module 714 can perform the aforementioned preprocessing on the simulation data, and then render the fitting effect based on the preprocessed simulation data. The process of the 3D rendering module 714 rendering the fitting effect can be referred to the aforementioned... Figure 6 The following is an introduction to steps S619~S621.

[0363] It should be noted that the electronic device 710 may also include more or fewer modules. For example, the electronic device 710 may also include the aforementioned modules. Figure 1 The electronic device 100 shown includes modules such as account management 211, product display 212, order placement 213, payment 214, and scene management 216D. The server 720 may also contain more or fewer modules. For details on the server 720, please refer to the aforementioned description of the server 200.

[0364] From the above Figure 7 As shown in the communication system 70, during the virtual try-on process, the electronic device 710 can reduce data transmission volume by sending human mesh parameters, try-on action ID, and clothing ID to the server 720, saving transmission bandwidth between the electronic device 710 and the server 720 and improving the efficiency of virtual try-on. The electronic device 710 can provide high-precision virtual try-on effects. When users purchase clothing online using the electronic device 710, they can conveniently view the try-on effect of a 3D image matching their body shape wearing the clothing, thus determining whether the clothing suits them. This can effectively reduce the situation where users buy unsuitable clothing online, lowering the return rate of clothing e-commerce.

[0365] Another communication system 80 provided in the embodiments of this application is described below.

[0366] Figure 8 An exemplary architecture diagram of a communication system 80 is shown. Figure 8 As shown, the communication system 80 may include an electronic device 810 and a server 820.

[0367] The electronic device 810 may include an action management module 811, a 3D image management module 812, a digital clothing simulation SDK 813, a 3D rendering module 814, and a scene management module 815. The digital clothing simulation SDK 813 may include an encoding / decoding module 813A, a network transmission module 813B, and a rendering preprocessing module 813C. The various modules included in the electronic device 810 can be referenced in the foregoing description. Figure 7 Introduction to the electronic device 710 shown.

[0368] Scene management module 815 is optional. In some embodiments, electronic device 810 may also omit scene management module 815.

[0369] The Digital Clothing Simulation SDK813 can be referred to as the processing module of the Electronic Device 810.

[0370] Server 820 may include a 3D digital clothing simulation module 821. The 3D digital clothing simulation module 821 may include a fabric simulation module 821A, an encoding / decoding module 821B, and a network transmission module 821C. The 3D digital clothing simulation module 821 can refer to the aforementioned... Figure 7 The following is an introduction to the 3D digital clothing simulation module 724 shown.

[0371] In some embodiments, the digital clothing simulation SDK 813 can acquire 3D image data from the 3D image management module 812 and skeletal data of the fitting action from the action management module 811. The digital clothing simulation SDK 813 can also acquire 3D digital clothing data of the garment to be tried on. This 3D digital clothing data can be acquired by the digital clothing simulation SDK 813 from the server 820, or from a server associated with an e-commerce application, or from the memory of the electronic device 810.

[0372] like Figure 8 As shown, the Digital Clothing Simulation SDK 813 can send 3D image data, skeletal data of fitting actions, and 3D digital clothing data to the 3D Digital Clothing Simulation Module 821 in the server 820.

[0373] The 3D digital clothing simulation module 821 can perform fabric simulation based on the received 3D image data, skeletal data of the fitting action, and 3D digital clothing data to obtain clothing simulation data. During the fabric simulation process, the 3D digital clothing simulation module 821 can also calculate the human mesh data of the user's 3D image during the fitting action. The fabric simulation process described above can be found in the preceding embodiments and will not be repeated here.

[0374] The 3D digital clothing simulation module 821 can send the clothing simulation data and the human mesh data of the person performing the fitting action to the digital clothing simulation SDK 813. In this way, the digital clothing simulation SDK 813 or the 3D rendering module 814 does not need to synchronize data. The 3D rendering module 814 can use the aforementioned simulation data and the human mesh data of the person performing the fitting action to render the fitting effect. The implementation of the fitting effect rendering by the 3D rendering module 814 can be referred to the description in the foregoing embodiments.

[0375] In some embodiments, the normal calculation of the simulation data can be performed by the server 820. This reduces the computational power consumption on the electronic device 810 side. Alternatively, the normal calculation of the simulation data can be performed by the electronic device 810. This reduces the transmission bandwidth between the electronic device 810 and the server 820.

[0376] It should be noted that the electronic device 810 may also include more or fewer modules. For example, the electronic device 810 may also include the aforementioned modules. Figure 1 The electronic device 100 shown includes modules such as account management 211, product display 212, order placement 213, payment 214, and scene management 216D. The server 820 may also contain more or fewer modules. For details on the server 820, please refer to the aforementioned description of the server 200.

[0377] In some embodiments, the above Figure 8 The communication system 80 shown can be applied to scenarios where there is insufficient trust between the electronic device 810 and the server 820. For example, even if the server 820 does not include a 3D image management module, an action management module, or a 3D digital clothing management module, the electronic device 810 and the server 820 can collaborate on an end-to-cloud basis using the communication system 80 to provide users with virtual try-on services.

[0378] Another communication system 90 provided in the embodiments of this application is described below.

[0379] Figure 9 An exemplary architecture diagram of a communication system 90 is shown. Figure 9 The communication system 90 may include an electronic device 910 and a server 920.

[0380] Electronic device 910 may include an action management module 911, a 3D image management module 912, a digital clothing simulation SDK 913, a 3D rendering module 914, and a scene management module 915. The digital clothing simulation SDK 913 may include an encoding / decoding module 913A, a network transmission module 913B, and a rendering preprocessing module 913C. The various modules included in electronic device 910 can be referenced in the foregoing description. Figure 7 Introduction to the electronic device 710 shown.

[0381] The scene management module 915 is optional. In some embodiments, the electronic device 910 may also omit the scene management module 915.

[0382] The Digital Clothing Simulation SDK 913 can be referred to as the processing module of the Electronic Device 910.

[0383] Server 920 may include a 3D digital clothing management module 921, an action management module 922, a 3D image management module 923, a 3D digital clothing simulation module 924, and a database 925. The 3D digital clothing simulation module 924 may include a fabric simulation module 924A, an encoding / decoding module 924B, and a network transmission module 924C.

[0384] Among them, database 925 can be used to store simulation data of clothing in the case of a combination of 3D image + clothing + fitting action.

[0385] In some embodiments, the 3D digital clothing simulation module 924 can store the simulation data of the clothing obtained through fabric simulation in the database 925, and record the combination of 3D image + clothing + fitting action corresponding to the simulation data in the database.

[0386] During the virtual try-on process, the digital clothing simulation SDK 913 can send human mesh parameters, try-on action ID, and clothing ID to the 3D digital clothing simulation module 924. Based on the received human mesh parameters, try-on action ID, and clothing ID, the 3D digital clothing simulation module 924 can search the database 925 for identical or similar combinations of 3D image + clothing + try-on action. The aforementioned identical 3D image + clothing + try-on action combination can represent a combination of identical 3D images (e.g., identical body size information), identical clothing (e.g., identical style and size), and identical try-on action. The aforementioned similar 3D image + clothing + try-on action combination can represent a combination of similar 3D images (e.g., the difference in body size information between 3D images is less than a preset threshold), identical clothing, and identical try-on action.

[0387] When no identical or similar combination of 3D image + clothing + fitting action is found in database 925, the 3D digital clothing simulation module 924 can use the received human mesh parameters, fitting action ID, and clothing ID to perform fabric simulation and obtain clothing simulation data. The method for performing fabric simulation described above can be found in the previous section. Figure 7 The illustrated embodiment is described below. Subsequently, the 3D digital clothing simulation module 924 can send the clothing simulation data to the digital clothing simulation SDK 913.

[0388] When the same or similar 3D image + clothing + fitting action combination is found in the database 925, the 3D digital clothing simulation module 924 can obtain historical simulation data (i.e., simulation data of the clothing corresponding to the same or similar 3D image + clothing + fitting action combination) from the database 925. The 3D digital clothing simulation module 924 can then send the historical simulation data to the digital clothing simulation SDK 913.

[0389] After receiving the clothing simulation data from the server 920, the electronic device 910 can render a virtual try-on effect. For details, please refer to the description in the foregoing embodiments; further elaboration will not be repeated here.

[0390] It should be noted that the electronic device 910 may also include more or fewer modules. For example, the electronic device 910 may also include the aforementioned modules. Figure 1 The electronic device 100 shown includes modules such as account management 211, product display 212, order placement 213, payment 214, and scene management 216D. The server 920 may also contain more or fewer modules. For details on the server 920, please refer to the aforementioned description of the server 200.

[0391] Depend on Figure 9 As shown in the communication system 90, the server 920 can store the simulation data of clothing obtained from fabric simulation. When it is necessary to perform the same fitting action on the same or similar 3D image and put on the same clothing for fabric simulation again, the server 920 can retrieve the clothing simulation data from the database 925 without having to perform the specific calculation process of fabric simulation again. This can reduce the repetitive fabric simulation process performed by the server 920, save the server 920's computing resources, and improve the concurrency capability of the server 920.

[0392] It is understood that the user interfaces described in the embodiments of this application are merely example interfaces and do not constitute a limitation on the solution of this application. In other embodiments, the user interface may adopt different interface layouts, may include more or fewer controls, and may add or remove other functional options, as long as they are based on the same inventive concept provided in this application, they are all within the protection scope of this application.

[0393] It should be noted that, without causing contradictions or conflicts, any feature in any embodiment of this application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of this application.

[0394] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A virtual try-on method, characterized in that, The method includes: The electronic device receives a first operation, which is used to select a first garment for virtual try-on. The electronic device sends the first human body data of the first 3D image, the first action data of the first action sequence, and the first clothing data of the first clothing to the server, wherein the first action sequence includes a first action and a second action. The electronic device receives first simulation data of the first garment from the server; The electronic device renders a first set of fitting effects based on the first simulation data. The first set of fitting effects includes a first fitting effect and a second fitting effect. The first fitting effect is used to represent the effect of the first 3D image wearing the first clothing and performing the first action. The second fitting effect is used to represent the effect of the first 3D image wearing the first clothing and performing the second action. The first fitting effect and the second fitting effect are different.

2. The method according to claim 1, characterized in that, The first human body data includes the first human body mesh data of the first 3D image, the first motion data includes the first skeleton data of the first motion and the second skeleton data of the second motion, and the first clothing data includes the clothing mesh data and clothing parameter information of the first clothing. The clothing parameter information includes one or more of the following: the stretch coefficient, bending coefficient, and friction coefficient of the first clothing. or, The first human body data includes human body parameters of the first 3D image, the human body parameters are used to generate the first human body mesh data, the first action data includes identification information of the first action and the second action, and the first clothing data includes identification information of the first clothing.

3. The method according to claim 1 or 2, characterized in that, Before the electronic device sends the first human body data of the first 3D image, the first action data of the first action sequence, and the first clothing data of the first garment to the server, the method further includes: The electronic device receives an operation to select the first 3D image and / or the first action sequence.

4. The method according to any one of claims 1-3, characterized in that, The first set of fitting effects is presented by playing a first video, and the method further includes: The electronic device receives a second operation for the first video, the second operation being used to play the first video, or to pause playback of the first video, or to fast forward the first video, or to rewind the first video; The electronic device controls the playback of the first video according to the second operation.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: The electronic device receives a third operation, which is used to select a second garment for virtual try-on. In response to the third operation, the electronic device sends second garment data and first layer information of the second garment to the server, the first layer information being used to indicate the layering of the first garment and the second garment; The electronic device receives second simulation data from the server; The electronic device renders a second set of fitting effects based on the second simulation data. The second set of fitting effects includes a third fitting effect and a fourth fitting effect. The third fitting effect is used to represent the effect of the first 3D image wearing the first garment and the second garment and performing the first action. The fourth fitting effect is used to represent the effect of the first 3D image wearing the first garment and the second garment and performing the second action.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: The electronic device receives a fourth operation, which is used to select a third garment to replace the first garment for virtual try-on. In response to the fourth operation, the electronic device sends the third garment data of the third garment to the server; The electronic device receives third simulation data of the third garment from the server; The electronic device renders a third set of fitting effects based on the third simulation data. The third set of fitting effects includes a fifth fitting effect and a sixth fitting effect. The fifth fitting effect is used to represent the effect of the first 3D image wearing the third clothing and performing the first action. The sixth fitting effect is used to represent the effect of the first 3D image wearing the third clothing and performing the second action.

7. The method according to any one of claims 1-4, characterized in that, The method further includes: The electronic device receives a fifth operation, which is used to change the first action sequence into a second action sequence, the second action sequence including a third action and a fourth action. In response to the fifth operation, the electronic device sends the second action data of the second action sequence to the server; The electronic device receives fourth simulation data of the first garment from the server; The electronic device renders a fourth set of fitting effects based on the fourth simulation data. The fourth set of fitting effects includes a seventh fitting effect and an eighth fitting effect. The seventh fitting effect is used to represent the effect of the first 3D image wearing the first clothing and performing the third action. The eighth fitting effect is used to represent the effect of the first 3D image wearing the first clothing and performing the fourth action.

8. The method according to any one of claims 1-4, characterized in that, The method further includes: The electronic device receives a sixth operation, which is used to select the fitting scene as the first scene; In response to the sixth operation, the electronic device acquires the first scene data of the first scene; The electronic device renders the first set of fitting effects based on the first simulation data, specifically including: The electronic device renders the first set of fitting effects based on the first scene data and the first simulation data. The first set of fitting effects includes the first fitting effect and the second fitting effect. The first fitting effect is used to represent the effect of the first 3D character wearing the first clothing and performing the first action in the first scene. The second fitting effect is used to represent the effect of the first 3D character wearing the first clothing and performing the second action in the first scene.

9. The method according to claim 2, characterized in that, The method further includes: The electronic device acquires first body size information, which includes one or more of the following: height, shoulder width, chest circumference, waist circumference, hip circumference, arm circumference, arm length, neck circumference, abdominal circumference, hip circumference, thigh circumference, calf circumference, and leg length. The electronic device obtains the first human mesh data of the first 3D image based on the first body size information; or, The electronic device acquires one or more images, and the one or more images display a human body corresponding to the first 3D image; The electronic device obtains the first human mesh data of the first 3D image based on one or more images.

10. A virtual try-on method, characterized in that, The method is executed by the processing module, and the method includes: Acquire first human body data of the first 3D avatar, first action data of the first action sequence, and first clothing data of the first clothing, wherein the first action sequence includes a first action and a second action; Send the first human body data, the first action data, and the first clothing data to the server; Receive first simulation data of the first garment from the server; The first simulation data is given to the 3D rendering module to render the first set of fitting effects. The first set of fitting effects includes the first fitting effect and the second fitting effect. The first fitting effect is used to show the effect of the first 3D image wearing the first clothing and performing the first action. The second fitting effect is used to show the effect of the first 3D image wearing the first clothing and performing the second action. The first fitting effect and the second fitting effect are different.

11. The method according to claim 10, characterized in that, The first human body data includes the first human body mesh data of the first 3D image, the first motion data includes the first skeleton data of the first motion and the second skeleton data of the second motion, and the first clothing data includes the clothing mesh data and clothing parameter information of the first clothing. The clothing parameter information includes one or more of the following: the stretch coefficient, bending coefficient, and friction coefficient of the first clothing. or, The first human body data includes human body parameters of the first 3D image, the human body parameters are used to generate the first human body mesh data, the first action data includes identification information of the first action and the second action, and the first clothing data includes identification information of the first clothing.

12. The method according to claim 10 or 11, characterized in that, The step of having the first simulation data rendered by the 3D rendering module to produce the first set of fitting effects specifically includes: The first simulation data is preprocessed to obtain preprocessed data. The preprocessing includes one or more of the following: calculating the normals of the first simulation data and synchronizing the first simulation data with the human body data of the first 3D image in the first action sequence. The preprocessed data is then used by the 3D rendering module to render the first set of fitting effects.

13. The method according to any one of claims 10-12, characterized in that, The method further includes: Acquire the second garment data and the first layer information of the second garment, wherein the first layer information is used to indicate the layering of the first garment and the second garment; Send the second clothing data and the first layer information to the server; Receive second simulation data from the server; The second simulation data is given to the 3D rendering module to render a second set of fitting effects. The second set of fitting effects includes a third fitting effect and a fourth fitting effect. The third fitting effect is used to represent the effect of the first 3D image wearing the first garment and the second garment and performing the first action. The fourth fitting effect is used to represent the effect of the first 3D image wearing the first garment and the second garment and performing the second action.

14. An electronic device, characterized in that, The electronic device includes a communication device, a memory, and a processor, wherein, The communication device is used to communicate with the server; The storage device is used to store computer programs; The processor is used to invoke the computer program, causing the electronic device to perform the method according to any one of claims 1-9.

15. A computationally readable storage medium, comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device performs the method of any one of claims 1-9.

16. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1-9.

Images