Virtual fashion adaptation method and device, computer program product and electronic equipment

Abstract

The invention provides a virtual fashion adaptation method and device, a computer program product and electronic equipment, and relates to the technical field of games. The method comprises the following steps: acquiring an original fashion model and a role model, wherein the original fashion model is not matched with the role model in shape; respectively adding three-dimensional mark points for the original fashion model and the role model, wherein the three-dimensional mark points of the original fashion model are in one-to-one correspondence with the three-dimensional mark points of the role model; based on the three-dimensional mark points of the original fashion model and the three-dimensional mark points of the role model, respectively constructing a skeleton of the original fashion model and a skeleton of the role model; based on the skeleton of the original fashion model, skeleton binding information of the original fashion model is generated, and the skeleton of the original fashion model is deformed by referring to the skeleton of the role model to obtain a deformed skeleton; based on the deformation skeleton and the skeleton binding information of the original fashion model, the original fashion model is deformed, the target fashion model matched with the shape of the role model is obtained, the cost is low, and the applicability is high.

Description

Technical Field

[0001] The present disclosure relates to the field of game technology, and in particular to a virtual fashion adaptation method, a virtual fashion adaptation device, a computer program product, and an electronic device. Background Art

[0002] In order to meet the aesthetic needs of different users and cater to the development of different in-game activities, it is usually necessary to design virtual characters of various body shapes. There may be a need for virtual characters of different body shapes to wear the same set of virtual clothing.

[0003] In the related art, manually modifying virtual fashion models to fit characters of different body shapes is labor-intensive and costly. The fashion adaptation solution based on deep learning needs to learn to express the parameters of the real human body through image information, but in games, due to the different styles of different games, the game character models are very different from the real human body and cannot be well applied in games.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a virtual fashion adaptation method, a virtual fashion adaptation device, a computer program product and an electronic device, so as to at least to some extent solve the problems of low cost and limited applicability in related technologies.

[0006] According to a first aspect of the present disclosure, a method for adapting virtual fashion is provided, the method comprising: obtaining an original fashion model and a role model, the original fashion model and the role model not matching in body shape; adding three-dimensional marker points to the original fashion model and the role model, respectively, the three-dimensional marker points of the original fashion model and the three-dimensional marker points of the role model corresponding one to one; constructing a skeleton of the original fashion model and a skeleton of the role model based on the three-dimensional marker points of the original fashion model and the three-dimensional marker points of the role model, respectively; generating bone binding information of the original fashion model based on the skeleton of the original fashion model, and deforming the skeleton of the original fashion model with reference to the skeleton of the role model to obtain a deformed skeleton; deforming the original fashion model based on the deformed skeleton and the bone binding information of the original fashion model to obtain a target fashion model matching in body shape with the role model.

[0007] According to a second aspect of the present disclosure, a virtual fashion adaptation device is provided, the device comprising: a model acquisition module, used to acquire an original fashion model and a role model, the original fashion model and the role model are not compatible in body shape; a marker point adding module, used to add three-dimensional marker points to the original fashion model and the role model, respectively, the three-dimensional marker points of the original fashion model and the three-dimensional marker points of the role model correspond one to one; a skeleton construction module, used to construct the skeleton of the original fashion model and the skeleton of the role model respectively based on the three-dimensional marker points of the original fashion model and the three-dimensional marker points of the role model; a skeleton deformation module, used to generate bone binding information of the original fashion model based on the skeleton of the original fashion model, and deform the skeleton of the original fashion model with reference to the skeleton of the role model to obtain a deformed skeleton; a fashion model deformation module, used to deform the original fashion model based on the deformed skeleton and the bone binding information of the original fashion model to obtain a target fashion model that is compatible with the body shape of the role model.

[0008] According to a third aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the virtual fashion adaptation method and possible implementation manner thereof of the first aspect are implemented.

[0009] According to a fourth aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the virtual fashion adaptation method and possible implementation methods of the above-mentioned first aspect by executing the executable instructions.

[0010] The technical solution disclosed in this disclosure has the following beneficial effects:

[0011] In the above-mentioned virtual fashion adaptation process, the original fashion model and the role model are obtained, and the body shapes of the original fashion model and the role model are not compatible; three-dimensional marker points are added to the original fashion model and the role model respectively, and the three-dimensional marker points of the original fashion model correspond to the three-dimensional marker points of the role model one by one; based on the three-dimensional marker points of the original fashion model and the three-dimensional marker points of the role model, the skeleton of the original fashion model and the skeleton of the role model are constructed respectively; based on the skeleton of the original fashion model, the bone binding information of the original fashion model is generated, and with reference to the skeleton of the role model, the skeleton of the original fashion model is deformed to obtain a deformed skeleton; based on the deformed skeleton and the bone binding information of the original fashion model, the original fashion model is deformed to obtain a target fashion model that is adapted to the body shape of the role model. The present invention reconstructs the skeletons of the fashion model and the character model, and deforms the skeleton of the original fashion model with reference to the skeleton of the character model, so that the original fashion model also undergoes corresponding deformation based on the skeleton deformation, thereby adapting to character models of different body shapes. This not only realizes the reuse of fashion resources and reduces the cost of model production, but also the process has no special requirements for the mesh topological structure of the fashion model and the character model, and has a wider applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A flowchart showing a method for fitting a virtual fashion in this exemplary embodiment is shown;

[0013] Figure 2A A schematic diagram showing an original fashion model in this exemplary embodiment;

[0014] Figure 2B A schematic diagram showing a role model in this exemplary embodiment;

[0015] Figure 3A A schematic diagram showing a method of marking joint points in a model view of an original fashion model in this exemplary embodiment is shown;

[0016] Figure 3B Another schematic diagram of marking joint points in a model view of an original fashion model in this exemplary embodiment is shown;

[0017] Figure 4 A flowchart of determining three-dimensional position information of joint points in a first to-be-processed model by setting a virtual camera in this exemplary embodiment is shown;

[0018] Figure 5 A schematic diagram of skeleton construction of an original fashion model in this exemplary embodiment is shown;

[0019] Figure 6 A schematic diagram showing a visualization of the right arm bone weight in this exemplary embodiment;

[0020] Figure 7 A flow chart of deforming the skeleton of an original fashion model in this exemplary embodiment is shown;

[0021] Figure 8 A flow chart showing a method of adjusting the limb radius of a bone to be processed in this exemplary embodiment is shown;

[0022] Fig. 9 A schematic diagram showing a dressed role model after merging a target fashion model with a role model in this exemplary embodiment;

[0023] Fig.10 A schematic diagram of a process of dressing a virtual character in this exemplary embodiment is shown;

[0024] Fig.11 A structural block diagram of a virtual fashion adaptation device in this exemplary embodiment is shown;

[0025] Fig.12 An electronic device for implementing the virtual fashion fitting method in this exemplary embodiment is shown. DETAILED DESCRIPTION

[0026] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings.

[0027] The accompanying drawings are schematic diagrams of the present disclosure and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in hardware modules or integrated circuits, or in networks, processors or microcontrollers. The embodiments can be implemented in various forms and should not be construed as being limited to the examples set forth herein. The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the description below, many specific details are provided to provide a full description of the embodiments of the present disclosure. However, those skilled in the art should appreciate that one or more specific details may be omitted when implementing the technical solution of the present disclosure, or one or more specific details may be replaced by other methods, components, devices, steps, etc.

[0028] In the related art, manually modifying virtual fashion models to fit characters of different body shapes is labor-intensive and costly. The fashion adaptation solution based on deep learning needs to learn to express the parameters of the real human body through image information, but in games, due to the different styles of different games, the game character models are very different from the real human body and cannot be well applied in games.

[0029] In view of one or more of the above problems, exemplary embodiments of the present disclosure provide a method for fitting a virtual fashion, an apparatus for fitting a virtual fashion, a computer program product, and an electronic device.

[0030] In an optional embodiment, reference Figure 1 As shown, a flowchart of a virtual fashion adaptation method is provided, which specifically includes the following steps S110 to S150:

[0031] Step S110, obtaining an original fashion model and a character model, wherein the original fashion model and the character model do not match each other in body shape;

[0032] Step S120, adding three-dimensional marking points to the original fashion model and the character model respectively, and the three-dimensional marking points of the original fashion model correspond to the three-dimensional marking points of the character model one by one;

[0033] Step S130, constructing a skeleton of the original fashion model and a skeleton of the character model based on the three-dimensional marking points of the original fashion model and the three-dimensional marking points of the character model;

[0034] Step S140, based on the skeleton of the original fashion model, generating the skeleton binding information of the original fashion model, and deforming the skeleton of the original fashion model with reference to the skeleton of the character model to obtain a deformed skeleton;

[0035] Step S150: deforming the original fashion model based on the deformable skeleton and the skeleton binding information of the original fashion model to obtain a target fashion model that matches the body shape of the character model.

[0036] Figure 1 In the method shown, the skeletons of the fashion model and the character model are reconstructed, and the skeleton of the original fashion model is deformed with reference to the skeleton of the character model, so that the original fashion model is also deformed accordingly based on the skeleton deformation, thereby adapting to character models of different body shapes. This not only realizes the reuse of fashion resources and reduces the cost of model production, but also the process has no special requirements for the mesh topological structure of the fashion model and the character model, and has a wider applicability.

[0037] Below Figure 1 Provide detailed instructions for each step.

[0038] In step S110, an original fashion model and a character model are obtained, and the body shapes of the original fashion model and the character model are not compatible.

[0039] The original fashion model may be a model that can change the appearance of a character, including but not limited to clothing, skin, etc. The original fashion model may be a virtual fashion mesh created in advance for dressing up a virtual character. If the fashion model is not compatible with the body shape of the character model to be dressed up, it can be deformed through the fashion model adaptation solution in the present disclosure to adapt to the character model to be dressed up.

[0040] For example, the original fashion model may be composed of a shawl, a top, a skirt, shoes, hair accessories, etc., which is not specifically limited in the present disclosure. Figure 2A As shown, a schematic diagram of an original fashion model is provided.

[0041] The role model may be a pre-created mesh of a virtual role to be dressed up, and the body shape of the role model may be reflected in height, weight, and body shape. Figure 2B As shown, a schematic diagram of a role model is provided.

[0042] In step S120, three-dimensional marking points are added to the original fashion model and the character model respectively, and the three-dimensional marking points of the original fashion model correspond to the three-dimensional marking points of the character model one by one.

[0043] The three-dimensional marker points may be model joint points. For example, the added three-dimensional marker points may include but are not limited to: shoulder marker points, neck marker points, elbow marker points, etc., which are not specifically limited in the present disclosure.

[0044] In an optional embodiment, the above-mentioned adding three-dimensional marker points to the original fashion model and the character model respectively can be achieved through the following steps: obtaining the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed based on the two-dimensional position information of the joint points in the first model to be processed; based on the three-dimensional position information of the joint points in the first model to be processed, adding three-dimensional marker points to the first model to be processed.

[0045] Among them, the first model to be processed is any one of the original fashion model and the character model, the two-dimensional position information of the joint points refers to the position information of the joint points in the two-dimensional plane, and the three-dimensional position information of the joint points refers to the position information of the joint points in the three-dimensional space.

[0046] Based on the three-dimensional position information of the joint points in the first model to be processed, three-dimensional marker points are added to the first model to be processed so as to construct a skeleton matching the body shape of the corresponding model.

[0047] In an optional embodiment, the above-mentioned obtaining of the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed based on the two-dimensional position information of the joint points in the first model to be processed, can be achieved through the following steps: based on the user's joint point marking operation in the model view of the first model to be processed, the two-dimensional position information of the marked joint points can be determined; based on the two-dimensional position information of the marked joint points, the three-dimensional position information of the marked joint points can be determined.

[0048] For example, Figure 3A and Figure 3B As shown, a schematic diagram of marking joint points in the model view of the original fashion model is provided to assist the user in adding three-dimensional marking points.

[0049] For example, after determining the two-dimensional position information of the marked joint point, the three-dimensional position information of the marked joint point can be obtained by finding the intersection of a straight line passing through the marked joint point and facing forward with the first model to be processed, wherein the straight line facing forward can be a straight line perpendicular to the model view plane.

[0050] Optionally, manual intervention may be performed to fine-tune the three-dimensional position information of the determined joint points to improve the accuracy of the three-dimensional marking points, which is not specifically limited in the present disclosure.

[0051] In an optional embodiment, the above-mentioned obtaining the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed based on the two-dimensional position information of the joint points in the first model to be processed, can also be achieved by the following steps: setting a plurality of virtual cameras surrounding the first model to be processed in the three-dimensional space, and photographing the first model to be processed by the virtual cameras to obtain a plurality of model images; identifying the two-dimensional position information of the joint points in each model image, and determining the three-dimensional position information of the identified joint points based on the two-dimensional position information of the identified joint points in each model image.

[0052] Taking the original fashion model as an example, N virtual cameras can be set in the three-dimensional space, and these N virtual cameras can be evenly distributed on the horizontal plane around the original fashion model, all facing the center of the original fashion model. Wherein, N is an integer, which can be set in advance, and the present disclosure does not specifically limit this.

[0053] Optionally, manual intervention may be performed to fine-tune the configuration of the virtual camera in order to improve the accuracy and adaptability of the shooting angle, which is not specifically limited in the present disclosure.

[0054] Exemplarily, based on neural network-based pose prediction algorithms, such as MMPose, OpenPose, etc., the positions of M joint points on each model image are identified to obtain the two-dimensional position information of each joint point in each model image. Here, M is an integer that can be preset in advance, and the present disclosure does not specifically limit this.

[0055] Optionally, based on the two-dimensional position information of the identified joint points in each model image, determining the three-dimensional position information of the identified joint points can be achieved through the following steps: Based on the two-dimensional position information of the identified joint points in each model image, starting from each virtual camera, multiple rays passing through the identified joint points are constructed in the three-dimensional space; the position information of the point with the minimum sum of distances from the multiple rays in the three-dimensional space is used as the three-dimensional position information of the identified joint points.

[0056] Exemplarily, for the j-th joint point, traverse its two-dimensional positions on each model image, starting from the origin of each virtual camera, construct rays passing through the joint points on the projection plane of each virtual camera, use the gradient descent method to find the point with the minimum sum of distances from these rays in the three-dimensional space, and use the position information of this point as the three-dimensional position information of the j-th joint point to facilitate improving the automation performance of marker addition.

[0057] As Figure 4 shown, a flowchart for determining the three-dimensional position information of joint points in the first model to be processed by setting virtual cameras is provided, which may specifically include the following steps:

[0058] Step S401, set the positions of N virtual cameras;

[0059] Step S402, let i = 1;

[0060] Step S403, determine whether i < N + 1 is satisfied; if yes, execute Step S404; if not, jump to Step S408;

[0061] Step S404, render the image Image[i] with the i-th virtual camera;

[0062] Step S405, use the pose prediction algorithm to analyze the image Image[i] to obtain the two-dimensional position information set {KP[i][j], 0 < j <= M} of M joint points on the projection plane of the virtual camera; where, KP[i][j] can represent the two-dimensional position coordinates of the j-th joint point on the projection plane of the i-th virtual camera;

[0063] Step S406: Construct a set of rays {Ray[i][j], 0 < j <= M} from the focus of the virtual camera to each position coordinate in the two-dimensional position information set {KP[i][j], 0 < j <= M}; where Ray[i][j] can represent the ray from the focus of the i-th virtual camera to the j-th joint point.

[0064] Step S407: Let i = i + 1, and jump to step S403.

[0065] Step S408: Let j = 1.

[0066] Step S409: Determine whether j < M + 1 is satisfied; if satisfied, execute step S410, if not satisfied, jump to step S412.

[0067] Step S410: Calculate the point in three-dimensional space with the minimum sum of distances from each ray in the set of rays {Ray[i][j], 0 < i <= N}, and store its three-dimensional position information in the position with subscript j in the array P.

[0068] Step S411: Let j = j + 1, and jump to step S409.

[0069] Step S412: Return the array P.

[0070] It should be noted that the character model can add three-dimensional marker points in the same way as the original fashion model to ensure that the three-dimensional marker points of the original fashion model and the character model are in one-to-one correspondence.

[0071] In step S130, based on the three-dimensional marker points of the original fashion model and the three-dimensional marker points of the character model, the skeleton of the original fashion model and the skeleton of the character model are respectively constructed.

[0072] Among them, the skeleton can be composed of multiple bones.

[0073] In an alternative embodiment, the above-mentioned construction of the skeleton of the original fashion model and the skeleton of the character model based on the three-dimensional marker points of the original fashion model and the three-dimensional marker points of the character model can be achieved through the following steps: Based on the preset marker point connection relationship, add bones to the three-dimensional marker points of the second model to be processed to obtain the skeleton of the second model to be processed.

[0074] Among them, the second model to be processed can be any one of the original fashion model and the character model.

[0075] Among them, the preset marker point connection relationship can be set in advance according to the actual situation. For example, the three-dimensional marker point of the shoulder is connected to the three-dimensional marker points of the neck and the elbow.

[0076] Specifically, bones can be added at the three-dimensional marking points of the second model to be processed according to the connection relationship of the preset marking points to obtain the skeleton of the second model to be processed, thereby realizing the creation of the skeleton, and the operation is simple and easy to implement.

[0077] It should be noted that the original fashion model and the character model can adopt the same skeleton creation method, so that the skeleton of the original fashion model and the skeleton of the character model have the same skeleton structure, that is, there is a one-to-one correspondence between the bones.

[0078] In an optional embodiment, the above-mentioned adding of bones to the three-dimensional marker points of the second model to be processed based on the preset marker point connection relationship can be achieved through the following steps: adding bones to the three-dimensional marker points to be processed, and determining whether there is a next three-dimensional marker point to the three-dimensional marker points to be processed; wherein the three-dimensional marker points to be processed are any three-dimensional marker points of the second model to be processed; if there is a next three-dimensional marker point to the three-dimensional marker points to be processed, rotating the bones to point to the next three-dimensional marker point of the three-dimensional marker points to be processed; if there is no next three-dimensional marker point to the three-dimensional marker points to be processed, rotating the bones to face away from the previous three-dimensional marker point of the three-dimensional marker points to be processed.

[0079] Specifically, the skeleton can be constructed by traversing each three-dimensional marking point of the second model to be processed and adding bones to the three-dimensional marking points of the second model to be processed.

[0080] Specifically, when adding bones to the 3D marker point to be processed, the starting point of the bones can be placed at the 3D marker point to be processed; if the 3D marker point to be processed has the next 3D marker point, the bones can be rotated so that the end point of the bones points to the next 3D marker point to be processed; if the 3D marker point to be processed does not have the next 3D marker point, the bones can be rotated so that the end point of the bones points away from the previous 3D marker point to be processed and keeps consistent with the direction of the upper bones, so as to quickly construct the skeleton of the second model to be processed. For example, Figure 5 As shown, a schematic diagram of the skeleton construction of an original fashion model is provided.

[0081] Optionally, the skeleton of the original fashion model and the skeleton of the character model constructed may have a tree topological relationship. Exemplarily, the skeleton located at the trunk position may be used as the root of the tree structure.

[0082] It should be noted that after the bone is added, the end point of the bone can be connected to the starting point of its child bones, and this constraint can be maintained during the operation of the bone. That is, when the bone is moved, rotated, or scaled to move its end point, the starting point of its child bones also moves with it.

[0083] In step S140, based on the skeleton of the original fashion model, the skeletal binding information of the original fashion model is generated, and with reference to the skeleton of the character model, the skeleton of the original fashion model is deformed to obtain a deformed skeleton.

[0084] The bone binding information may include but is not limited to the bones corresponding to the model vertices and their bone weights.

[0085] In an optional embodiment, the bone binding information includes the bone weights of the model vertices. The above-mentioned generation of the bone binding information of the original fashion model based on the skeleton of the original fashion model can be achieved through the following steps: for each bone in the skeleton of the original fashion model, the bone weight of each voxel of the original fashion model is determined; and according to the bone weights of each voxel of the original fashion model, the bone weights of each model vertex of the original fashion model are determined.

[0086] Optionally, voxels may be constructed using a mesh of the original fashion model. Exemplarily, a rasterization algorithm may be used for voxelization, and the voxel grid resolution may be configured as 256×256×256 by default.

[0087] Specifically, for each bone in the skeleton of the original fashion model, the weight of each voxel of the original fashion model for the corresponding bone can be determined, and based on the weight of each voxel of the original fashion model for the corresponding bone, the weight of each vertex of the original fashion model for the corresponding bone can be determined, so as to obtain the bone binding information of the original fashion model.

[0088] Optionally, the bone weights of each voxel of the original fashion model can be determined by voxel heat diffusion method. Specifically, the above determination of the bone weights of each voxel of the original fashion model can be achieved by the following steps: determining the voxels with bones passing through and the voxels without bones passing through from each voxel of the original fashion model; configuring different initial bone weights for the voxels with bones passing through and the voxels without bones passing through; and repeatedly calculating the initial bone weights based on a preset number of updates. To update the initial bone weight of each voxel in the original fashion model, the final bone weight of each voxel in the original fashion model is obtained; where k is an integer, which is used to represent the current update number of the target voxel, W k-1 represents the bone weight before the target voxel is updated, W k represents the updated bone weight of the target voxel, lr represents the learning rate, Represents the mean value of the target voxel and the voxels adjacent to the target voxel; the target voxel is the voxel of the original fashion model.

[0089] Exemplarily, voxels through which bones pass and voxels through which bones do not pass can be determined from each voxel of the original fashion model; an initial bone weight of 1 is configured for voxels through which bones pass, and an initial bone weight of 0 is configured for voxels through which bones do not pass.

[0090] Exemplarily, the difference between each voxel and the mean of all its neighboring voxels (including itself) can be calculated, and the new bone weight can be obtained by subtracting the product of the difference and the learning rate lr from its own bone weight.

[0091] Exemplarily, the lr parameter may be set to 0.1.

[0092] Optionally, the target voxel may be a voxel that intersects with the mesh of the original fashion model and does not have a voxel through which the bone passes. Exemplarily, if the voxels adjacent to a voxel do not intersect with the corresponding model mesh, its bone weight may be regarded as 0 and may not participate in the weight update. Exemplarily, if there is a voxel through which the bone passes, its bone weight may be regarded as 1 and may not participate in the weight update to reduce unnecessary computational overhead.

[0093] Optionally, the above-mentioned determination of the bone weights of each model vertex of the original fashion model based on the bone weights of each voxel of the original fashion model can be achieved by the following steps: using the bone weights of the voxels where each model vertex of the original fashion model is located as the bone weights of each model vertex of the original fashion model.

[0094] For example, Figure 6 As shown, a schematic diagram is provided to visualize the weights of the right arm bones.

[0095] Optionally, for each bone in the skeleton of the original fashion model, after determining the bone weights of each vertex of the original fashion model, the weights may be normalized. Normalize the bone weights. Q is the total number of bones in the skeleton of the original fashion model, and W′ m Represents the bone weight of the model vertex for bone m, W′ x Represents the bone weight of the model vertex for bone x.

[0096] In an optional embodiment, the skeleton of the original fashion model is deformed with reference to the character model to obtain a deformed skeleton, which can be achieved through the following steps: keeping the starting point position of the root bone in the skeleton of the original fashion model consistent with the starting point position of the root bone in the skeleton of the character model; taking the root bone in the skeleton of the original fashion model as the bone to be processed; determining the difference vector according to the end point position of the bone to be processed and the end point position of the reference bone corresponding to the bone to be processed; wherein the reference bone corresponding to the bone to be processed is the bone in the skeleton of the character model; if the length of the difference vector is greater than or equal to a preset threshold, adjusting the rotation and scaling parameters of the bone to be processed according to the difference vector; if the length of the difference vector is less than the preset threshold, updating the bone to be processed according to the brother bones or child bones of the bone to be processed, until all bones in the skeleton of the original fashion model are traversed to obtain the deformed skeleton.

[0097] For example, the bone located at the torso position in the skeleton of the original fashion model can be used as the root bone.

[0098] The difference vector can be used to measure the difference between the bone to be processed and the reference bone. It should be noted that the bone can be regarded as a vector pointing from one point to another point, the length of the vector can be regarded as the length of the bone, and the direction of the vector can be regarded as the direction of the bone. Therefore, the difference vector between the bone to be processed and the reference bone can be determined according to the end point position of the bone to be processed and the end point position of the reference bone corresponding to the bone to be processed.

[0099] The preset threshold value can be set according to actual needs, and the present disclosure does not specifically limit this. It is understandable that the smaller the preset threshold value is set, the closer the skeleton of the original fashion model is to the skeleton of the character model after deformation. Exemplarily, the preset threshold value can be configured as 0.001.

[0100] Optionally, when adjusting the rotation scaling parameters of the bone to be processed according to the difference vector, the gradient descent method can be used for adjustment. Exemplarily, the rotation scaling parameters of the bone to be processed can be added with a minimum value h, at which time the stop point of the bone to be processed will move, and the difference vector is recalculated based on the stop point after the bone to be processed moves to obtain an increment Δf of the difference vector, and the rotation scaling parameters of the bone to be processed are subtracted by Δf / h to update the rotation scaling parameters of the bone to be processed, and the degree of difference between the bone to be processed and the reference bone is reduced by continuous iterative updating. Specifically, the iterative updating can be stopped when the difference vector is less than a preset threshold value. Exemplarily, the minimum value can be configured to be 0.001, and the present disclosure does not specifically limit this.

[0101] For example, Figure 7 As shown, a flowchart for deforming the skeleton of an original fashion model is provided, which may specifically include the following steps:

[0102] Step S701, start;

[0103] Step S702, moving the starting position of the root bone in the skeleton of the original fashion model to a position corresponding to the starting position of the root bone in the skeleton of the character model;

[0104] Step S703, taking the root bone in the skeleton of the original fashion model as the bone to be processed;

[0105] Step S704, determine whether the skeleton to be processed is empty; if not, execute step S705; if yes, execute step S709;

[0106] Step S705, calculating the difference vector between the end point of the bone to be processed and the end point of the reference bone corresponding to the bone to be processed;

[0107] Step S706, determine whether the length of the difference vector is less than a preset threshold; if not, execute step S707; if yes, execute step S708;

[0108] Step S707, adding a minimum value h to the rotation and scaling parameters of the bone to be processed, at which time the end point of the bone to be processed will move, recalculating the difference vector according to the end point after the movement of the bone to be processed, obtaining an increment Δf of the difference vector, and subtracting Δf / h from the rotation and scaling parameters of the bone to be processed; jumping to step S705;

[0109] Step S708, taking the next brother bone of the bone to be processed as the new bone to be processed, if there is no next brother bone, taking the first child bone of the bone to be processed as the new bone to be processed; jump to step S704;

[0110] Step S709, completed.

[0111] Optionally, in addition to making the starting points of each skeleton of the fashion model and the character model consistent, the thickness of the limbs of the fashion model and the character model are made consistent to enhance the fashion adaptability: determine the limb radius of the bone to be processed and the limb radius of the reference bone corresponding to the bone to be processed; adjust the limb radius of the bone to be processed with reference to the limb radius of the reference bone corresponding to the bone to be processed.

[0112] Exemplarily, the limb radius of the bone to be processed can be determined by the following steps: radially emitting multiple rays from the starting point of the bone to be processed, and determining the intersection points of these rays with the fashion model, and taking the average of the distances between these intersection points and the starting point of the bone to be processed as the limb radius of the bone to be processed.

[0113] Exemplarily, the limb radius of the reference bone can be determined by the following steps: radially emitting multiple rays from the starting point of the reference bone, and determining the intersection points of these rays with the character model, and taking the average of the distances between these intersection points and the starting point of the reference bone as the limb radius of the reference bone.

[0114] For example, Figure 8 As shown, a flowchart for adjusting the limb radius of the bone to be processed is provided, which may specifically include the following steps:

[0115] Step S801, start;

[0116] Step S802, moving the starting point position of the root bone in the skeleton of the original fashion model to a position corresponding to the starting point position of the root bone in the skeleton of the character model;

[0117] Step S803, taking the root bone in the skeleton of the original fashion model as the bone to be processed;

[0118] Step S804, determine whether the skeleton to be processed is empty; if not, execute step S805; if yes, execute step S809;

[0119] Step S805, emitting multiple rays radially from the starting point of the bone to be processed, and determining the intersection points of these rays with the fashion model, and taking the average of the distances between these intersection points and the starting point of the bone to be processed as the limb radius Ra of the bone to be processed;

[0120] Step S806, emitting multiple rays radially from the starting point of the reference skeleton, and determining the intersection points of these rays with the character model, and taking the average of the distances between these intersection points and the starting point of the reference skeleton as the limb radius Rb of the reference skeleton;

[0121] Step S807, multiplying the limb radius of the bone to be processed by Rb / Ra to adjust the limb radius of the bone to be processed;

[0122] Step S808, taking the next brother bone of the bone to be processed as the new bone to be processed, if there is no next brother bone, taking the first child bone of the bone to be processed as the new bone to be processed; jump to step S804;

[0123] Step S809, completed.

[0124] In step S150, the original fashion model is deformed based on the deformation skeleton and the skeleton binding information of the original fashion model to obtain a target fashion model that matches the body shape of the character model.

[0125] The target fashion model is a deformed model of the original fashion model.

[0126] For example, based on the skeleton binding information of the original fashion model, the deformation generated by the skeleton can be applied to the corresponding mesh model to obtain the deformed target fashion model. The original fashion model is deformed. Among them, P rest Indicates the initial position of the mesh vertices of the original fashion model; W k is the weight for bone k; T k is the bone state transformation matrix; M k The default matrix for bones; Indicates M k The inverse matrix of transform Indicates the position of the mesh vertices of the original fashion model after deformation.

[0127] It should be noted that a skeleton can be represented as a set of matrices that can be bound to vertices on a mesh to control the deformation of the mesh. A single skeleton can be composed of multiple bones, each of which can be represented as a matrix. Each bone has a default matrix M and can also have an externally set transformation matrix T to control the deformation of the mesh.

[0128] After deformation, the mesh body shape corresponding to the original fashion model (ie, the target fashion model) can be consistent with the mesh body shape corresponding to the character model.

[0129] Optionally, the deformed model data of the original fashion model can be written into the mesh vertices of the original fashion model. In this case, the original fashion model can maintain the deformed shape without bones, and the bones of the original fashion model can be directly deleted and no longer used.

[0130] In an optional implementation, the following steps may also be performed: merging the target fashion model with the role model to obtain a dressed role model.

[0131] The dressed role model refers to the model after the role model is dressed.

[0132] Exemplarily, the model polygons corresponding to the target fashion model and the model polygons corresponding to the character model may be taken as a union to obtain the dressed character model, thereby achieving the adaptation of the fashion model to different body types.

[0133] like Fig. 9 As shown, a schematic diagram of a dressed role model after merging a target fashion model and a role model is provided.

[0134] In an optional implementation, before merging the target fashion model with the character model, the following steps may be performed to further enhance the matching degree between the fashion model and the character model: determining the part of the character model that conflicts with the target fashion model; and deleting the conflicting part of the character model.

[0135] For example, the same part in the character model as that in the target fashion model can be used as the conflicting part. For example, if both the fashion model and the character model have sleeves, the sleeve part in the character model can be deleted to avoid affecting the effect after wearing the clothes.

[0136] Optionally, the part of the target fashion model that is the same as the character model may be regarded as the conflicting part, and the conflicting part of the target fashion model may be deleted to avoid affecting the effect after dressing.

[0137] like Fig.10 As shown, a schematic diagram of a process of dressing a virtual character is provided, which may specifically include the following steps:

[0138] Step S1001, obtaining an original fashion model and a character model, wherein the original fashion model and the character model do not match each other in body shape;

[0139] Step S1002, adding three-dimensional marking points to the original fashion model and the character model respectively, and the three-dimensional marking points of the original fashion model correspond to the three-dimensional marking points of the character model one by one;

[0140] Step S1003, constructing the skeleton of the original fashion model and the skeleton of the character model respectively based on the 3D marking points of the original fashion model and the 3D marking points of the character model;

[0141] Step S1004, based on the skeleton of the original fashion model, generating the skeleton binding information of the original fashion model, and deforming the skeleton of the original fashion model with reference to the skeleton of the character model to obtain a deformed skeleton;

[0142] Step S1005, deforming the original fashion model based on the deformable skeleton and the skeleton binding information of the original fashion model to obtain a target fashion model that matches the body shape of the character model;

[0143] Step S1006, determining the part of the character model that conflicts with the target fashion model, and deleting the part of the character model that conflicts;

[0144] Step S1007: merge the target fashion model and the character model to obtain a dressed character model.

[0145] The exemplary embodiment of the present disclosure also provides a virtual fashion fitting device, referring to Fig.11 As shown, the virtual fashion adaptation device 1100 may include the following program modules:

[0146] The model acquisition module 1110 is used to acquire the original fashion model and the role model, and the body shapes of the original fashion model and the role model are not compatible;

[0147] The marking point adding module 1120 is used to add three-dimensional marking points to the original fashion model and the character model respectively, and the three-dimensional marking points of the original fashion model correspond to the three-dimensional marking points of the character model one by one;

[0148] A skeleton construction module 1130 is used to construct the skeleton of the original fashion model and the skeleton of the role model respectively based on the three-dimensional marking points of the original fashion model and the three-dimensional marking points of the role model;

[0149] The skeleton deformation module 1140 is used to generate the bone binding information of the original fashion model based on the skeleton of the original fashion model, and deform the skeleton of the original fashion model with reference to the skeleton of the character model to obtain a deformed skeleton;

[0150] The fashion model deformation module 1150 is used to deform the original fashion model based on the deformation skeleton and the bone binding information of the original fashion model to obtain a target fashion model that matches the body shape of the character model.

[0151] In an optional embodiment, based on the aforementioned scheme, the marker point adding module 1120 includes: a joint position determination module, used to obtain the two-dimensional position information of the joint points in the first model to be processed, and determine the three-dimensional position information of the joint points in the first model to be processed based on the two-dimensional position information of the joint points in the first model to be processed; wherein the first model to be processed is any one of the original fashion model and the character model; a joint marking module, used to add three-dimensional marker points to the first model to be processed based on the three-dimensional position information of the joint points in the first model to be processed.

[0152] In an optional embodiment, based on the aforementioned scheme, the joint position determination module can be configured to: determine the two-dimensional position information of the marked joint points based on the user's joint point marking operation in the model view of the first model to be processed; determine the three-dimensional position information of the marked joint points based on the two-dimensional position information of the marked joint points.

[0153] In an optional embodiment, based on the aforementioned scheme, the joint position determination module includes: an image shooting module, which is used to set up multiple virtual cameras surrounding the first model to be processed in three-dimensional space, and shoot the first model to be processed through the virtual cameras to obtain multiple model images; a three-dimensional position determination module, which is used to identify the two-dimensional position information of the joint point in each model image, and determine the three-dimensional position information of the identified joint point based on the two-dimensional position information of the identified joint point in each model image.

[0154] In an optional embodiment, based on the aforementioned scheme, the three-dimensional position determination module determines the three-dimensional position information of the identified joint points based on the two-dimensional position information of the identified joint points in each model image, and can be configured as follows: based on the two-dimensional position information of the identified joint points in each model image, taking each virtual camera as the starting point, constructing multiple rays passing through the identified joint points in the three-dimensional space; and taking the position information of the point in the three-dimensional space with the smallest sum of distances to the multiple rays as the three-dimensional position information of the identified joint points.

[0155] In an optional embodiment, based on the aforementioned scheme, the skeleton construction module 1130 includes: a skeleton adding module, which is used to add skeletons to the three-dimensional marker points of the second model to be processed based on the preset marker point connection relationship to obtain the skeleton of the second model to be processed; wherein the second model to be processed is any one of the original fashion model and the character model.

[0156] In an optional embodiment, based on the above-mentioned scheme, the skeleton adding module can be configured to: add skeletons to the three-dimensional marker point to be processed, and determine whether there is a next three-dimensional marker point to the three-dimensional marker point to be processed; wherein the three-dimensional marker point to be processed is any three-dimensional marker point of the second model to be processed; if there is a next three-dimensional marker point to the three-dimensional marker point to be processed, rotate the skeleton to point to the next three-dimensional marker point of the three-dimensional marker point to be processed; if there is no next three-dimensional marker point to the three-dimensional marker point to be processed, rotate the skeleton to face away from the previous three-dimensional marker point of the three-dimensional marker point to be processed.

[0157] In an optional implementation, based on the above scheme, the bone binding information includes the bone weights of the model vertices, and the skeleton deformation module 1140 includes: a bone binding information determination module, which is used to generate the bone binding information of the original fashion model based on the skeleton of the original fashion model. The bone binding information determination module includes: a bone weight determination module, which is used to determine the bone weight of each voxel of the original fashion model for each bone in the skeleton of the original fashion model, and determine the bone weight of each model vertex of the original fashion model according to the bone weight of each voxel of the original fashion model.

[0158] In an optional embodiment, based on the above scheme, the bone weight determination module determines the bone weight of each voxel of the original fashion model, including: determining the voxels through which the bones pass and the voxels through which the bones do not pass from the voxels of the original fashion model; configuring different initial bone weights for the voxels through which the bones pass and the voxels through which the bones do not pass respectively; and repeatedly calculating the initial bone weights based on a preset number of updates. To update the initial bone weight of each voxel in the original fashion model, the final bone weight of each voxel in the original fashion model is obtained; where k represents the current update number of the target voxel, W k-1 represents the bone weight before the target voxel is updated, W k represents the updated bone weight of the target voxel, lr represents the learning rate, Represents the mean value of the target voxel and the voxels adjacent to the target voxel; the target voxel is the voxel of the original fashion model.

[0159] In an optional embodiment, based on the aforementioned scheme, the bone weight determination module determines the bone weights of each model vertex of the original fashion model according to the bone weights of each voxel of the original fashion model, including: using the bone weights of the voxels where each model vertex of the original fashion model is located as the bone weights of each model vertex of the original fashion model.

[0160] In an optional embodiment, based on the above scheme, the skeleton deformation module 1140 includes: a skeleton parameter adjustment module. The skeleton parameter adjustment module can be configured to: keep the starting position of the root bone in the skeleton of the original fashion model consistent with the starting position of the root bone in the skeleton of the character model; use the root bone in the skeleton of the original fashion model as the bone to be processed; determine the difference vector according to the end point position of the bone to be processed and the end point position of the reference bone corresponding to the bone to be processed; wherein the reference bone corresponding to the bone to be processed is the bone in the skeleton of the character model; if the length of the difference vector is greater than or equal to the preset threshold, adjust the rotation and scaling parameters of the bone to be processed according to the difference vector; if the length of the difference vector is less than the preset threshold, update the bone to be processed according to the brother bones or child bones of the bone to be processed, until all bones in the skeleton of the original fashion model are traversed to obtain the deformed skeleton.

[0161] In an optional implementation, based on the above solution, the virtual fashion adaptation device 1100 further includes: a limb radius adjustment module. The limb radius adjustment module can be configured to: determine the limb radius of the bone to be processed and the limb radius of the reference bone corresponding to the bone to be processed; and adjust the limb radius of the bone to be processed with reference to the limb radius of the reference bone corresponding to the bone to be processed.

[0162] In an optional implementation, based on the above solution, the virtual fashion adaptation device 1100 further includes: a model merging module, which is used to merge the target fashion model with the role model to obtain a dressed role model.

[0163] In an optional implementation, based on the above solution, before merging the target fashion model with the role model, the virtual fashion adaptation device 1100 further includes: a conflicting part deletion module. The conflicting part deletion module can be configured to: determine the part of the role model that conflicts with the target fashion model; and delete the conflicting part of the role model.

[0164] The specific details of each part of the above-mentioned device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0165] The exemplary embodiments of the present disclosure also provide a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the above-mentioned virtual fashion adaptation method is implemented.

[0166] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0167] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0168] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0169] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure. Exemplarily, the following steps may be included:

[0170] The original fashion model and the character model are obtained, and the body shapes of the original fashion model and the character model are not compatible;

[0171] Add three-dimensional marker points to the original fashion model and the character model respectively, and the three-dimensional marker points of the original fashion model correspond to the three-dimensional marker points of the character model one by one;

[0172] Based on the 3D marker points of the original fashion model and the 3D marker points of the character model, respectively constructing the skeleton of the original fashion model and the skeleton of the character model;

[0173] Based on the skeleton of the original fashion model, bone binding information of the original fashion model is generated, and with reference to the skeleton of the character model, the skeleton of the original fashion model is deformed to obtain a deformed skeleton;

[0174] Based on the deformable skeleton and the bone binding information of the original fashion model, the original fashion model is deformed to obtain a target fashion model that matches the body shape of the character model.

[0175] In an optional embodiment, based on the aforementioned scheme, the above-mentioned three-dimensional marker points are added to the original fashion model and the character model respectively, including: obtaining the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed according to the two-dimensional position information of the joint points in the first model to be processed; wherein the first model to be processed is any one of the original fashion model and the character model; based on the three-dimensional position information of the joint points in the first model to be processed, adding three-dimensional marker points to the first model to be processed.

[0176] In an optional embodiment, based on the aforementioned scheme, the above-mentioned two-dimensional position information of the joint points in the first model to be processed is obtained, and the three-dimensional position information of the joint points in the first model to be processed is determined according to the two-dimensional position information of the joint points in the first model to be processed, including: based on the user's joint point marking operation in the model view of the first model to be processed, determining the two-dimensional position information of the marked joint points; based on the two-dimensional position information of the marked joint points, determining the three-dimensional position information of the marked joint points.

[0177] In an optional embodiment, based on the aforementioned scheme, the above-mentioned obtaining of the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed according to the two-dimensional position information of the joint points in the first model to be processed, includes: setting a plurality of virtual cameras surrounding the first model to be processed in the three-dimensional space, and photographing the first model to be processed by the virtual cameras to obtain a plurality of model images; identifying the two-dimensional position information of the joint points in each model image, and determining the three-dimensional position information of the identified joint points based on the two-dimensional position information of the identified joint points in each model image.

[0178] In an optional embodiment, based on the aforementioned scheme, the above-mentioned three-dimensional position information of the identified joint points is determined based on the two-dimensional position information of the identified joint points in each model image, including: based on the two-dimensional position information of the identified joint points in each model image, taking each virtual camera as the starting point, constructing multiple rays passing through the identified joint points in the three-dimensional space; taking the position information of the point in the three-dimensional space with the smallest sum of distances to the multiple rays as the three-dimensional position information of the identified joint points.

[0179] In an optional embodiment, based on the aforementioned scheme, the above-mentioned three-dimensional marker points of the original fashion model and the three-dimensional marker points of the character model are used to construct the skeleton of the original fashion model and the skeleton of the character model respectively, including: based on the preset marker point connection relationship, at the three-dimensional marker points of the second model to be processed, bones are added to obtain the skeleton of the second model to be processed; wherein the second model to be processed is any one of the original fashion model and the character model.

[0180] In an optional embodiment, based on the aforementioned scheme, the above-mentioned method adds bones to the three-dimensional marker points of the second model to be processed based on the preset marker point connection relationship, including: adding bones to the three-dimensional marker points to be processed, and determining whether there is a next three-dimensional marker point at the three-dimensional marker point to be processed; wherein the three-dimensional marker point to be processed is any three-dimensional marker point of the second model to be processed; if there is a next three-dimensional marker point at the three-dimensional marker point to be processed, rotating the bones to point to the next three-dimensional marker point of the three-dimensional marker point to be processed; if there is no next three-dimensional marker point at the three-dimensional marker point to be processed, rotating the bones to face away from the previous three-dimensional marker point of the three-dimensional marker point to be processed.

[0181] In an optional embodiment, based on the aforementioned scheme, the bone binding information includes the bone weights of the model vertices. The above-mentioned bone binding information of the original fashion model is generated based on the skeleton of the original fashion model, including: for each bone in the skeleton of the original fashion model, determining the bone weights of each voxel of the original fashion model, and determining the bone weights of each model vertex of the original fashion model according to the bone weights of each voxel of the original fashion model.

[0182] In an optional embodiment, based on the above scheme, the above-mentioned determination of the bone weights of each voxel of the original fashion model includes: determining the voxels through which bones exist and the voxels through which bones do not exist from the voxels of the original fashion model; configuring different initial bone weights for the voxels through which bones exist and the voxels through which bones do not exist; and repeatedly calculating the initial bone weights based on a preset number of updates. To update the initial bone weight of each voxel in the original fashion model, the final bone weight of each voxel in the original fashion model is obtained; where k represents the current update number of the target voxel, W k-1 represents the bone weight before the target voxel is updated, W k represents the updated bone weight of the target voxel, lr represents the learning rate, Represents the mean value of the target voxel and the voxels adjacent to the target voxel; the target voxel is the voxel of the original fashion model.

[0183] In an optional embodiment, based on the aforementioned scheme, the bone weights of each model vertex of the original fashion model are determined according to the bone weights of each voxel of the original fashion model, including: using the bone weights of the voxels where each model vertex of the original fashion model is located as the bone weights of each model vertex of the original fashion model.

[0184] In an optional embodiment, based on the aforementioned scheme, the skeleton of the original fashion model is deformed with reference to the skeleton of the character model to obtain a deformed skeleton, including: keeping the starting point position of the root bone in the skeleton of the original fashion model consistent with the starting point position of the root bone in the skeleton of the character model; using the root bone in the skeleton of the original fashion model as the bone to be processed; determining a difference vector according to the end point position of the bone to be processed and the end point position of the reference bone corresponding to the bone to be processed; wherein the reference bone corresponding to the bone to be processed is a bone in the skeleton of the character model; if the length of the difference vector is greater than or equal to a preset threshold, adjusting the rotation and scaling parameters of the bone to be processed according to the difference vector; if the length of the difference vector is less than the preset threshold, updating the bone to be processed according to the brother bones or child bones of the bone to be processed, until all bones in the skeleton of the original fashion model are traversed to obtain a deformed skeleton.

[0185] In an optional embodiment, based on the above-mentioned scheme, the following steps can also be performed: determine the limb radius of the bone to be treated and the limb radius of the reference bone corresponding to the bone to be treated; adjust the limb radius of the bone to be treated with reference to the limb radius of the reference bone corresponding to the bone to be treated.

[0186] In an optional implementation, based on the above solution, the following steps may also be performed: merging the target fashion model with the role model to obtain a dressed role model.

[0187] In an optional implementation, based on the above solution, before merging the target fashion model with the character model, the following steps may be performed: determining a portion of the character model that conflicts with the target fashion model; and deleting the portion of the character model that conflicts.

[0188] In the above steps, the skeletons of the fashion model and the character model are reconstructed, and the skeleton of the original fashion model is deformed with reference to the skeleton of the character model, so that the original fashion model is also deformed accordingly based on the skeleton deformation, thereby adapting to character models of different body shapes. This not only realizes the reuse of fashion resources and reduces the cost of model production, but also the process has no special requirements for the mesh topological structure of the fashion model and the character model, and has a wider applicability.

[0189] The exemplary embodiment of the present disclosure also provides an electronic device capable of implementing the above-mentioned virtual fashion adaptation method. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program code. The processor executes the method in the exemplary embodiment by executing the executable instructions. In addition, the electronic device may also include a display for displaying a graphical user interface.

[0190] Reference below Fig.12 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Fig.12 The electronic device 1200 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0191] like Fig.12 As shown, the electronic device 1200 may include: a processor 1210 , a memory 1220 , a bus 1230 , an I / O (input / output) interface 1240 , a network adapter 1250 , and a display 1260 .

[0192] The memory 1220 may include a volatile memory, such as a RAM 1221, a cache unit 1222, and may also include a non-volatile memory, such as a ROM 1223. The memory 1220 may also include one or more program modules 1224, such program modules 1224 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 1224 may include each module in the above-mentioned device.

[0193] The processor 1210 may include one or more processing units. For example, the processor 1210 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0194] The processor 1210 may be configured to execute executable instructions stored in the memory 1220 , such as executing any one or more method steps in the exemplary embodiment.

[0195] Exemplarily, the processor 1210 may perform the following steps:

[0196] The original fashion model and the character model are obtained, and the body shapes of the original fashion model and the character model are not compatible;

[0197] Add three-dimensional marker points to the original fashion model and the character model respectively, and the three-dimensional marker points of the original fashion model correspond to the three-dimensional marker points of the character model one by one;

[0198] Based on the 3D marker points of the original fashion model and the 3D marker points of the character model, respectively constructing the skeleton of the original fashion model and the skeleton of the character model;

[0199] Based on the skeleton of the original fashion model, bone binding information of the original fashion model is generated, and with reference to the skeleton of the character model, the skeleton of the original fashion model is deformed to obtain a deformed skeleton;

[0200] Based on the deformable skeleton and the bone binding information of the original fashion model, the original fashion model is deformed to obtain a target fashion model that matches the body shape of the character model.

[0201] In an optional embodiment, based on the aforementioned scheme, the above-mentioned three-dimensional marker points are added to the original fashion model and the character model respectively, including: obtaining the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed according to the two-dimensional position information of the joint points in the first model to be processed; wherein the first model to be processed is any one of the original fashion model and the character model; based on the three-dimensional position information of the joint points in the first model to be processed, adding three-dimensional marker points to the first model to be processed.

[0202] In an optional embodiment, based on the aforementioned scheme, the above-mentioned two-dimensional position information of the joint points in the first model to be processed is obtained, and the three-dimensional position information of the joint points in the first model to be processed is determined according to the two-dimensional position information of the joint points in the first model to be processed, including: based on the user's joint point marking operation in the model view of the first model to be processed, determining the two-dimensional position information of the marked joint points; based on the two-dimensional position information of the marked joint points, determining the three-dimensional position information of the marked joint points.

[0203] In an optional embodiment, based on the aforementioned scheme, the above-mentioned obtaining of the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed according to the two-dimensional position information of the joint points in the first model to be processed, includes: setting a plurality of virtual cameras surrounding the first model to be processed in the three-dimensional space, and photographing the first model to be processed by the virtual cameras to obtain a plurality of model images; identifying the two-dimensional position information of the joint points in each model image, and determining the three-dimensional position information of the identified joint points based on the two-dimensional position information of the identified joint points in each model image.

[0204] In an optional embodiment, based on the aforementioned scheme, the above-mentioned three-dimensional position information of the identified joint points is determined based on the two-dimensional position information of the identified joint points in each model image, including: based on the two-dimensional position information of the identified joint points in each model image, taking each virtual camera as the starting point, constructing multiple rays passing through the identified joint points in the three-dimensional space; taking the position information of the point in the three-dimensional space with the smallest sum of distances to the multiple rays as the three-dimensional position information of the identified joint points.

[0205] In an optional embodiment, based on the aforementioned scheme, the above-mentioned three-dimensional marker points of the original fashion model and the three-dimensional marker points of the character model are used to construct the skeleton of the original fashion model and the skeleton of the character model respectively, including: based on the preset marker point connection relationship, at the three-dimensional marker points of the second model to be processed, bones are added to obtain the skeleton of the second model to be processed; wherein the second model to be processed is any one of the original fashion model and the character model.

[0206] In an optional embodiment, based on the aforementioned scheme, the above-mentioned method adds bones to the three-dimensional marker points of the second model to be processed based on the preset marker point connection relationship, including: adding bones to the three-dimensional marker points to be processed, and determining whether there is a next three-dimensional marker point at the three-dimensional marker point to be processed; wherein the three-dimensional marker point to be processed is any three-dimensional marker point of the second model to be processed; if there is a next three-dimensional marker point at the three-dimensional marker point to be processed, rotating the bones to point to the next three-dimensional marker point of the three-dimensional marker point to be processed; if there is no next three-dimensional marker point at the three-dimensional marker point to be processed, rotating the bones to face away from the previous three-dimensional marker point of the three-dimensional marker point to be processed.

[0207] In an optional embodiment, based on the aforementioned scheme, the bone binding information includes the bone weights of the model vertices. The above-mentioned bone binding information of the original fashion model is generated based on the skeleton of the original fashion model, including: for each bone in the skeleton of the original fashion model, determining the bone weights of each voxel of the original fashion model, and determining the bone weights of each model vertex of the original fashion model according to the bone weights of each voxel of the original fashion model.

[0208] In an optional embodiment, based on the above scheme, the above-mentioned determination of the bone weights of each voxel of the original fashion model includes: determining the voxels through which bones exist and the voxels through which bones do not exist from the voxels of the original fashion model; configuring different initial bone weights for the voxels through which bones exist and the voxels through which bones do not exist; and repeatedly calculating the initial bone weights based on a preset number of updates. To update the initial bone weight of each voxel in the original fashion model, the final bone weight of each voxel in the original fashion model is obtained; where k represents the current update number of the target voxel, W k-1 represents the bone weight before the target voxel is updated, Wk represents the updated bone weight of the target voxel, lr represents the learning rate, Represents the mean value of the target voxel and the voxels adjacent to the target voxel; the target voxel is the voxel of the original fashion model.

[0209] In an optional embodiment, based on the aforementioned scheme, the bone weights of each model vertex of the original fashion model are determined according to the bone weights of each voxel of the original fashion model, including: using the bone weights of the voxels where each model vertex of the original fashion model is located as the bone weights of each model vertex of the original fashion model.

[0210] In an optional embodiment, based on the aforementioned scheme, the skeleton of the original fashion model is deformed with reference to the skeleton of the character model to obtain a deformed skeleton, including: keeping the starting point position of the root bone in the skeleton of the original fashion model consistent with the starting point position of the root bone in the skeleton of the character model; using the root bone in the skeleton of the original fashion model as the bone to be processed; determining a difference vector according to the end point position of the bone to be processed and the end point position of the reference bone corresponding to the bone to be processed; wherein the reference bone corresponding to the bone to be processed is a bone in the skeleton of the character model; if the length of the difference vector is greater than or equal to a preset threshold, adjusting the rotation and scaling parameters of the bone to be processed according to the difference vector; if the length of the difference vector is less than the preset threshold, updating the bone to be processed according to the brother bones or child bones of the bone to be processed, until all bones in the skeleton of the original fashion model are traversed to obtain a deformed skeleton.

[0211] In an optional embodiment, based on the above-mentioned scheme, the following steps can also be performed: determine the limb radius of the bone to be treated and the limb radius of the reference bone corresponding to the bone to be treated; adjust the limb radius of the bone to be treated with reference to the limb radius of the reference bone corresponding to the bone to be treated.

[0212] In an optional implementation, based on the above solution, the following steps may also be performed: merging the target fashion model with the role model to obtain a dressed role model.

[0213] In an optional implementation, based on the above solution, before merging the target fashion model with the character model, the following steps may be performed: determining a portion of the character model that conflicts with the target fashion model; and deleting the portion of the character model that conflicts.

[0214] In the above steps, the skeletons of the fashion model and the character model are reconstructed, and the skeleton of the original fashion model is deformed with reference to the skeleton of the character model, so that the original fashion model is also deformed accordingly based on the skeleton deformation, thereby adapting to character models of different body shapes. This not only realizes the reuse of fashion resources and reduces the cost of model production, but also the process has no special requirements for the mesh topological structure of the fashion model and the character model, and has a wider applicability.

[0215] The bus 1230 is used to realize the connection between different components of the electronic device 1200, and may include a data bus, an address bus, and a control bus.

[0216] The electronic device 1200 can communicate with one or more external devices 1300 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 1240 .

[0217] The electronic device 1200 can communicate with one or more networks through the network adapter 1250. For example, the network adapter 1250 can provide mobile communication solutions such as 3G / 4G / 5G, or provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 1250 can communicate with other modules of the electronic device 1200 through the bus 1230.

[0218] The electronic device 1200 may display a graphical user interface and the like through the display 1260 .

[0219] although Fig.12 Not shown, other hardware and / or software modules may also be provided in the electronic device 1200, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID (Redundant Arrays of Independent Disks) system, a tape drive, and a data backup storage system.

[0220] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0221] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module" or "system". Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0222] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A virtual fashion adaptation method, characterized in that: The method comprises: Acquire an original fashion model and a character model, wherein the original fashion model is not compatible with the body shape of the character model; Adding three-dimensional marking points to the original fashion model and the character model respectively, wherein the three-dimensional marking points of the original fashion model correspond one-to-one to the three-dimensional marking points of the character model; Based on the three-dimensional marking points of the original fashion model and the three-dimensional marking points of the character model, respectively constructing a skeleton of the original fashion model and a skeleton of the character model; Based on the skeleton of the original fashion model, bone binding information of the original fashion model is generated, and with reference to the skeleton of the character model, the skeleton of the original fashion model is deformed to obtain a deformed skeleton; Based on the deformable skeleton and the bone binding information of the original fashion model, the original fashion model is deformed to obtain a target fashion model that matches the body shape of the character model.

2. The method according to claim 1, characterized in that The adding three-dimensional marking points to the original fashion model and the character model respectively includes: Acquire the two-dimensional position information of the joint points in the first model to be processed, and determine the three-dimensional position information of the joint points in the first model to be processed according to the two-dimensional position information of the joint points in the first model to be processed; wherein the first model to be processed is any one of the original fashion model and the character model; Based on the three-dimensional position information of the joint points in the first model to be processed, three-dimensional marking points are added to the first model to be processed.

3. The method according to claim 2, characterized in that The acquiring the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed according to the two-dimensional position information of the joint points in the first model to be processed, comprises: Based on a joint point marking operation performed by a user in a model view of a first model to be processed, determining two-dimensional position information of the marked joint point; Based on the two-dimensional position information of the marked joint points, the three-dimensional position information of the marked joint points is determined.

4. The method according to claim 2, characterized in that: The acquiring the two-dimensional position information of the joint points in the first model to be processed, and determining the three-dimensional position information of the joint points in the first model to be processed according to the two-dimensional position information of the joint points in the first model to be processed, comprises: Setting a plurality of virtual cameras surrounding a first model to be processed in a three-dimensional space, and photographing the first model to be processed by the virtual cameras to obtain a plurality of model images; The two-dimensional position information of the joint point in each model image is identified, and the three-dimensional position information of the identified joint point is determined based on the two-dimensional position information of the identified joint point in each model image.

5. The method according to claim 4, characterized in that The step of determining the three-dimensional position information of the identified joint point based on the two-dimensional position information of the identified joint point in each model image includes: Based on the two-dimensional position information of the identified joint points in each model image, taking each of the virtual cameras as a starting point, constructing a plurality of rays passing through the identified joint points in three-dimensional space; The position information of the point in the three-dimensional space whose sum of distances from the multiple rays is the smallest is used as the three-dimensional position information of the identified joint point.

6. The method according to claim 1, characterized in that The constructing the skeleton of the original fashion model and the skeleton of the role model respectively based on the three-dimensional marking points of the original fashion model and the three-dimensional marking points of the role model comprises: Based on the preset marking point connection relationship, bones are added to the three-dimensional marking points of the second model to be processed to obtain the skeleton of the second model to be processed; wherein the second model to be processed is any one of the original fashion model and the character model.

7. The method according to claim 6, characterized in that The adding of bones at the three-dimensional marking points of the second to-be-processed model based on the preset marking point connection relationship includes: Adding a skeleton to the three-dimensional mark point to be processed, and determining whether the three-dimensional mark point to be processed has a next three-dimensional mark point; wherein the three-dimensional mark point to be processed is any three-dimensional mark point of the second model to be processed; If there is a next three-dimensional marking point of the three-dimensional marking point to be processed, rotating the skeleton to point to the next three-dimensional marking point of the three-dimensional marking point to be processed; If there is no next three-dimensional marking point of the three-dimensional marking point to be processed, the skeleton is rotated to face away from the previous three-dimensional marking point of the three-dimensional marking point to be processed.

8. The method according to claim 1, characterized in that The bone binding information includes the bone weights of the model vertices. The generating the bone binding information of the original fashion model based on the skeleton of the original fashion model includes: For each bone in the skeleton of the original fashion model, the bone weight of each voxel of the original fashion model is determined, and according to the bone weight of each voxel of the original fashion model, the bone weight of each model vertex of the original fashion model is determined.

9. The method according to claim 8, characterized in that Determining the bone weight of each voxel of the original fashion model includes: Determining voxels where bones pass through and voxels where bones do not pass through from each voxel of the original fashion model; Different initial bone weights are configured for voxels with bones passing through them and voxels without bones passing through them; Repeat the calculation based on the preset number of updates to update the initial bone weight of each voxel in the original fashion model to obtain the final bone weight of each voxel in the original fashion model; Among them, k represents the current update number of the target voxel, W k-1 represents the bone weight before the target voxel is updated, W k represents the updated bone weight of the target voxel, lr represents the learning rate, represents the average value of the target voxel and the voxels adjacent to the target voxel; the target voxel is a voxel of the original fashion model.

10. The method according to claim 8, characterized in that The step of determining the bone weight of each vertex of the original fashion model according to the bone weight of each voxel of the original fashion model comprises: The bone weights of the voxels where the vertices of each model of the original fashion model are located are used as the bone weights of the vertices of each model of the original fashion model.

11. The method according to claim 1, characterized in that: The step of deforming the skeleton of the original fashion model with reference to the skeleton of the character model to obtain a deformed skeleton comprises: Keeping the starting point position of the root bone in the skeleton of the original fashion model consistent with the starting point position of the root bone in the skeleton of the character model; Using the root bone in the skeleton of the original fashion model as the bone to be processed; Determine a difference vector according to the end point position of the bone to be processed and the end point position of the reference bone corresponding to the bone to be processed; wherein the reference bone corresponding to the bone to be processed is a bone in the skeleton of the character model; If the length of the difference vector is greater than or equal to a preset threshold, adjusting the rotation and scaling parameters of the bone to be processed according to the difference vector; If the length of the difference vector is less than the preset threshold, the bone to be processed is updated according to the brother bones or child bones of the bone to be processed until each bone in the skeleton of the original fashion model is traversed to obtain a deformed skeleton.

12. The method according to claim 11, characterized in that The method further comprises: Determine the limb radius of the bone to be processed and the limb radius of the reference bone corresponding to the bone to be processed; Adjust the limb radius of the bone to be processed with reference to the limb radius of the reference bone corresponding to the bone to be processed.

13. The method according to claim 1, characterized in that The method further comprises: The target fashion model is combined with the role model to obtain a dressed role model.

14. The method according to claim 13, characterized in that Before merging the target fashion model with the character model, the method further includes: determining a portion of the character model that conflicts with the target fashion model; Delete the conflicting parts of the role model.

15. A virtual fashion fitting device, characterized in that: The device comprises: A model acquisition module, used to acquire an original fashion model and a role model, wherein the original fashion model is not compatible with the body shape of the role model; A marking point adding module, used to add three-dimensional marking points to the original fashion model and the character model respectively, and the three-dimensional marking points of the original fashion model correspond to the three-dimensional marking points of the character model one by one; A skeleton construction module, used to construct the skeleton of the original fashion model and the skeleton of the role model respectively based on the three-dimensional marking points of the original fashion model and the three-dimensional marking points of the role model; A skeleton deformation module, used to generate bone binding information of the original fashion model based on the skeleton of the original fashion model, and deform the skeleton of the original fashion model with reference to the skeleton of the character model to obtain a deformed skeleton; The fashion model deformation module is used to deform the original fashion model based on the deformation skeleton and the bone binding information of the original fashion model to obtain a target fashion model that matches the body shape of the character model.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

17. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 14 by executing the executable instructions.

Images