A virtual fitting method, system, device and storage medium

Abstract

The present invention discloses a virtual fitting method, including: making a three-dimensional model of clothing according to a two-dimensional image of clothing; fitting the three-dimensional clothing model to a three-dimensional standard human body model; obtaining a two-dimensional image of a target human body; inputting the obtained three-dimensional human body parameters into the standard human body model for fitting; matching the clothing three-dimensional model to the three-dimensional human body model; making the three-dimensional human body model and the target human body posture basically consistent. The present invention provides a method for obtaining accurate three-dimensional human body model parameters by analyzing a full-body photo of a human body through a deep neural network, and only an ordinary photo is needed to quickly model a human body; at the same time, the three-dimensional clothing model is obtained by processing a two-dimensional clothing picture in advance, and the user does not need to participate in these behind-the-scenes work, but only needs to select the clothing style that he wants to virtually fit, and the system will automatically match the corresponding clothing model, and maintain the authenticity and restoration through a series of methods.

Description

Technical Field

[0001] The present invention belongs to the field of user virtual dressing and fitting, and specifically relates to human body modeling, clothing modeling, and the fitting of clothing models with human body models used in virtual dressing, especially a matching method, system, device and storage medium for a custom human body model generated by extracting relevant information from photos based on machine learning and a three-dimensional clothing model. Background Art

[0002] With the development of Internet technology, online shopping has become more and more popular. Compared with shopping in physical stores, online shopping has the advantages of a wide variety of goods and convenient shopping. However, there are also some problems that are not easy to solve when buying goods online. The most important one is that you cannot check the goods you want to buy in person. Among all the types of goods, this problem is most prominent in clothing goods. Compared with shopping in physical stores, you can change clothes and check the effect of clothes in real time. Online clothing shopping cannot provide effect pictures for consumers themselves, but can only provide pictures of models trying on clothes. Some even have no pictures of trying on clothes at all. Consumers cannot intuitively obtain the degree of matching between clothing and their own body image in real time. This has caused a large number of returns and exchanges.

[0003] In response to this problem, operators try to use virtual fitting technology to provide consumers with simulated fitting effects to solve this problem. Of course, there are other occasions in reality where virtual dressing technology can be used, such as in online games. Therefore, this technology has developed rapidly.

[0004] Virtual fitting refers to a technology application that allows users to view the "dressing" effect in real time on the terminal screen without actually changing into the clothes they want to see. Existing dressing technology applications mainly include flat fitting and three-dimensional virtual fitting technology. The former basically collects user pictures and clothes pictures, then stretches or compresses the clothes to the same size as the human body, and then cuts and splices them to form a "dressed" image. However, due to the simple and crude image processing method, this type of image has poor authenticity and does not take into account the user's actual body shape at all. The clothes are just forcibly applied to the user's photos, which cannot meet the user's needs. The latter usually collects the three-dimensional information of the person through a three-dimensional acquisition device and synthesizes it in combination with the characteristics of the clothes, or manually inputs the body data information provided by the user, and generates a virtual three-dimensional human body model mesh according to certain rules, and then combines it with the clothing map. Overall, this type of three-dimensional virtual fitting requires a large amount of data collection or three-dimensional data calculation, with high hardware costs, and is not easy to promote among ordinary users.

[0005] With the development of cloud computing technology, artificial intelligence technology and intelligent terminal processing capabilities, two-dimensional virtual fitting technology has emerged. This type of technology mainly includes three steps: (1) processing the personal body information provided by the user to obtain the target human body model; (2) processing the clothing information to obtain the clothing model; (3) fusing the human body model and the clothing model to generate a simulated image of a person wearing the clothing.

[0006] Regarding point (1), due to the accumulation of many uncertain factors such as process design, model parameter selection, and neural network training methods, the quality of the final generated clothing-changing pictures is not as good as the traditional three-dimensional virtual fitting technology. Among them, the establishment of the human body model is its basic step, and the subsequent dressing process must also be based on the previously generated human body model. Therefore, once the human body model is generated inaccurately, it is easy to cause problems such as a large difference in body shape between the human body model and the person being fitted, loss of skin texture, loss of body parts, etc., which will affect the final generated clothing-changing picture effect.

[0007] In the field of image processing, 3D reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of 3D objects. It is the basis for processing, operating and analyzing the properties of 3D objects in a computer environment. It is also a key technology for establishing virtual reality in computers to express the objective world. It is widely used in computer animation, virtual reality, industrial inspection and other fields.

[0008] In the general field of computer vision, there are many starting points for human body modeling, usually including the use of 3D scanning equipment to perform all-round scanning of real human bodies, three-dimensional reconstruction methods based on multi-view depth of field photography, and three-dimensional reconstruction methods based on given images combined with human models. Among them, the use of 3D scanning equipment to perform all-round scanning of real human bodies can obtain the most and most accurate information, but such equipment is usually expensive and requires a high degree of cooperation from human models. The entire processing process has very high requirements for the processing equipment, so it is generally applied to some professional fields; secondly, the multi-view three-dimensional reconstruction method needs to provide multiple overlapping images of the reconstructed human body from multiple perspectives and establish a spatial conversion relationship between images. Using multiple groups of cameras, taking multiple photos, and stitching together a 3D model, the operation is relatively simplified, but the computational complexity is still large, and in most cases, only people participating on site can obtain multi-angle pictures. The model obtained by stitching the texture obtained by the multi-angle photography method of the depth camera does not have body scale data and cannot provide a basis for 3D perception. Again, the method of combining a single image with a human body model only requires one image. The method of intelligently generating three-dimensional human body characteristic curves based on a neural network can obtain weights and thresholds that can be used to describe the curves of the neck, chest, waist, and hips of the human body through neural network training. Then, based on the dimensional parameter information of the girth, width, thickness, etc. of the human body section, a three-dimensional human body curve that matches the real human body shape can be directly generated to obtain a predicted human body model. However, this method has a small amount of input information, and the solution process still requires a large amount of calculation, resulting in an unsatisfactory final model effect.

[0009] Regarding point (2), there are several different methods in the existing technology for generating three-dimensional clothing models. At present, the more traditional way of building three-dimensional clothing models is based on the design and stitching method of two-dimensional clothing pieces. This method requires certain clothing expertise to design the sample, which is not a quality possessed by all virtual fitting users. At the same time, this method also requires manual specification of the stitching relationship between the samples, which will consume a lot of time for setting. In addition, another relatively new three-dimensional modeling method is based on hand-drawing, which can generate a simple clothing model through the line information hand-drawn by the user. However, this method requires professional personnel to hand-draw, and has poor reproducibility and repeatability. It takes a lot of time for users to draw the details of the clothing, and it is difficult to promote it to e-commerce on a large scale. Both methods are more inclined to innovate and design new clothing rather than to perform three-dimensional modeling on existing clothing for sale. Another method is to obtain clothing picture information, and use image processing technology and graphic simulation technology in combination to finally generate a virtual three-dimensional clothing model. The contour and size of the clothing are obtained through contour detection and classification in the picture, and the key points of the edges are found from the contour through machine learning methods. The stitching information is generated through the correspondence between the key points. Finally, the physical stitching of the clothing is simulated in three-dimensional space to obtain the real effect of the clothing being worn on the human body.

[0010] Regarding point (3), the common virtual fitting rooms on the market currently focus on style matching, and do not directly simulate the natural properties of the collision between virtual characters and clothing fabrics, so there is still a great lack of realism. At present, more and more manufacturers use virtual characters to vividly express user postures, and simulate the collision response between clothing fabrics and human bodies in real time and render them in real time to increase the adhesion between the virtual world and the real world, bringing more dressing fun to virtual fitting users and allowing more people to enjoy the convenience brought by buying clothes.

[0011] In summary, based on Internet technology and the characteristics of the network environment in which it is located, the method of directly outputting the final image or photo after changing clothes from a single human body image is undoubtedly the most preferred and convenient method. Users do not need to be present at the scene in person, and only need a photo to complete the entire virtual dressing process. Then the question that follows is that as long as the resulting photo effect can be guaranteed to be basically equivalent to the real 3D simulated dressing effect, it will become the mainstream. Among them, (1) how to obtain a human body model that is closest to the real state of the human body through a photo, and (2) how to put a three-dimensional clothing model on the human body model in the state closest to the real state, have become the two most important and unavoidable problems in the virtual dressing method.

[0012] Regarding the first point. In the prior art, there are usually several methods for constructing human body models: (1) regression-based methods, which reconstruct a human body model represented by voxels through a convolutional neural network. The algorithm first estimates the positions of the main joints of the human body based on the input image, and then estimates the positions of the key points in a voxel grid of a given size, and then uses the entire shape of the occupied voxels to describe the reconstructed human body shape based on whether each unit voxel inside it is occupied; (2) human body reconstruction based on a single image, which estimates the three-dimensional shape and posture of the human body at the same time. This method first roughly marks the simple human skeleton key points on the image, and then performs initial matching and fitting of the human body model based on these rough key points to obtain the approximate shape of the human body. (3) 23 bone nodes are used to represent the human skeleton, and then the rotation of each bone node is used to represent the posture of the entire human body. At the same time, 6890 vertex positions are used to express the human body shape. In the fitting process, the positions of the bone nodes are given, and the parameters of the shape and posture are fitted at the same time, so as to perform three-dimensional human body reconstruction; or first use the CNN model to predict the key points on the image, and then use the SMPL model for fitting to obtain the initial human body model. Next, the fitted shape parameters are used to regress a human joint bounding box. Each joint corresponds to a bounding box, and its bounding box is represented by the axis length and radius. Finally, the initial model and the bounding box obtained by regression are combined to obtain a 3D human reconstruction. The above methods have the problems of slow modeling speed, insufficient modeling accuracy, and strong dependence of the reconstruction effect on the created body and posture database.

[0013] Prior art 1 discloses a human body modeling method based on body measurement data, such as Figure 1 As shown, the method includes: obtaining physical measurement data; according to the physical measurement data, performing linear regression on a pre-created human body model through a pre-trained prediction model, fitting to obtain a predicted human body model, the pre-created human body model includes a plurality of pre-defined groups of marked feature points and a corresponding standard shape basis, and the physical measurement data includes measurement data corresponding to each group of marked feature points; according to the predicted human body model, a target human body model is obtained, and the target human body model includes measurement data, a target shape basis, and a target shape coefficient. However, this method has very high requirements for physical measurement data, including body length data and girth data, such as height, arm length, shoulder width, leg length, calf length, thigh length, foot length, head circumference, chest circumference, waist circumference, thigh circumference, etc., which are not only measured but also calculated. It does save the amount of calculation, but the user experience is very bad and the program is very cumbersome. In addition, in the training of the human body model, the training method of the SMPL model is referenced.

[0014] The SMPL model is a parametric human model and a human modeling method proposed by the Max Planck Institute. This method can perform arbitrary human modeling and animation driving. The biggest difference between this method and traditional LBS is the method of human posture image surface morphology proposed by it. This method can simulate the convexity and concavity of human muscles during limb movement. Therefore, it can avoid surface distortion of the human body during movement and accurately depict the morphology of human muscle stretching and contraction. In this method, β and θ are input parameters, where β represents 10 parameters such as the proportion of a person's height, weight, head-to-body ratio, etc., and θ represents 75 parameters representing the overall motion posture of the human body and the relative angles of 24 joints. However, the core of this model generation method is the accumulation of a large amount of training data to obtain the relationship between body shape and shape basis. However, since this relationship has a strong correlation with each other, each shape basis cannot be independently controlled and is not easy to decouple. For example, there is a certain correlation between arms and legs. When the arms move, the legs will theoretically move with them. It is difficult to achieve improvements for different body shapes on the SMPL model.

[0015] Prior art 2 discloses a 3D human modeling method based on a single photo, including: obtaining a photo, parsing the photo, marking the key points of the human body in the photo, and calculating the spatial coordinates of the key points; obtaining the distance between the skeletal points in the pre-created standard human body model and the key points in the photo, aligning the skeletal points and the key points to generate a basic human body model; obtaining the basic texture in the pre-created standard human body model, performing difference calculation between the basic texture and the skin texture of the face in the photo, and using the edge channel for fusion to generate basic texture data; generating a 3D human body model based on the basic human body model and the basic texture data. 3D human body modeling is achieved through a photo, and the model is supported by the skeleton and muscle system, which can produce expressions and movements. However, this method is to match the distance between the key points of the user's photo and the key points of the standard mannequin, and then adjust the distance to achieve the posture of the target human body, and then perform difference calculation and fusion between the basic texture and the skin texture in the photo to obtain the final human body model. This method has a simple process and a small amount of calculation, but the accuracy and authenticity of the generated human body model are not very high.

[0016] Prior art three discloses a method for generating a three-dimensional human body model, including: obtaining a two-dimensional human body image; inputting the two-dimensional human body image into a three-dimensional human body parameter model to obtain three-dimensional human body parameters corresponding to the two-dimensional human body image; inputting the training sample into a neural network for training to obtain a three-dimensional human body parameter model, including: inputting the standard two-dimensional human body image in the training sample into the neural network to obtain predicted three-dimensional human body parameters corresponding to the standard two-dimensional human body image; adjusting the three-dimensional flexible deformable model according to the predicted three-dimensional human body parameters to obtain a predicted three-dimensional human body model; obtaining the predicted joint point positions in the standard two-dimensional human body image by reverse mapping according to the joint point positions in the predicted three-dimensional human body model. This modeling method uses model judgment and the parameters finally output by the neural network to only have node parameters, and then uses the mature body shape of the SMPL model to make detailed adjustments to it consistent with the target human body posture. Although the amount of calculation is reduced, due to the small number of input parameters and the fact that the adjustment can only be completed on the basis of the SMPL prediction model, it is difficult to output a particularly ideal human body model that is highly consistent with the target human body posture.

[0017] Regarding the second point. There is such a virtual fitting solution in the prior art, including: obtaining a dressed reference human body model and an undressed target human body model; embedding skeletons of the same hierarchical structure for the reference human body model and the target human body model respectively; performing skin binding on the skeletons of the reference human body model and the target human body model; calculating the rotation amount of the bones in the skeleton of the target human body model, and recursively adjusting all the bones in the skeleton of the target human body model so that the posture of the skeleton of the target human body model is consistent with that of the skeleton of the reference human body model; using the LBS skinning algorithm to deform the skin of the target human body model according to the rotation amount of the bones in the skeleton of the target human body model; and migrating the clothing model from the reference human body model to the target human body model on the basis of the skin deformation of the target human body model. This invention can reduce the difficulty of migrating the clothing model from the reference human body to the target human body after the postures of the target human body model and the reference human body model are adjusted to be consistent, and convert the inefficient non-rigid registration problem into an efficient rigid registration problem, thereby realizing the migration of the clothing model from the reference human body model to the target human body model. The technical problem of completing the automatic fitting of clothes on different human bodies and in different postures while keeping the size of the clothes unchanged before and after fitting is solved. However, this skinning method pays too much attention to the fixed distance between clothes and skin, which has an advantage in fitting speed, but has a big disadvantage in clothing matching and realism. It is only suitable for some occasions that need to quickly and easily process the movement of clothes following the skin mesh.

[0018] It can be seen that in order to keep up with the development trend of the Internet industry, in the field of virtual fitting, the least input information, the least amount of calculation and the best effect will be the three basic goals that have been pursued. We need to find an optimal balance between these three and provide a virtual fitting method that can achieve simple input, the amount of calculation does not exceed the tolerance of the terminal device, and the effect is close to that of professional equipment. Summary of the invention

[0019] Based on the above problems, the present invention provides a virtual fitting method, system, device and storage medium that overcome the above problems.

[0020] The present invention provides a virtual fitting method, which comprises: obtaining a two-dimensional image of clothing; making a three-dimensional model of the clothing according to the two-dimensional image of the clothing; constructing a three-dimensional standard human body model in combination with a mathematical model, wherein the three-dimensional standard model is in an initial posture; fitting the three-dimensional clothing model to a three-dimensional standard human body model in an initial posture, that is, a standard basic mannequin; obtaining a two-dimensional image of a target human body; obtaining three-dimensional target human body model parameters by calculating a neural network model; inputting several groups of obtained three-dimensional human body parameters of posture and body shape into the three-dimensional standard human body model for fitting; and obtaining a target human body model having the same posture and body shape as the target human body and wearing the changed clothing.

[0021] Preferably, the three-dimensional standard human body model can be a three-dimensional standard human body model in other predetermined postures except the initial T-pose; initial posture bases of several common postures are pre-designed, and in the process of moving the three-dimensional human body model, first calculate which common posture has the shortest distance to the target posture, and then choose to start fitting from the initial posture base of this common posture to improve the processing speed.

[0022] Preferably, the three-dimensional standard human body model is composed of parameters of several body bases and several root bone parameters, the several bases constitute the entire human body model, and each body base is controlled by a parameter separately without affecting each other; the three-dimensional human body model has a mathematical weight relationship between bone points and human body mesh, and the determination of bone points can be associated with the human body model of the target human body posture.

[0023] Preferably, the method further includes two different skin brush weight processing steps between the three-dimensional clothing model and the skeleton, and between the human body model skin mesh and the skeleton, so as to ensure an improved motion matching relationship between the clothing, the human body skin mesh and the skeleton.

[0024] Preferably, the step of obtaining the three-dimensional target human body model parameters also includes: substituting the two-dimensional human body contour image into a first neural network that has undergone deep learning to perform joint point regression, so as to obtain a joint point map, a semantic segmentation map, body bone points and key point information of the target human body; and substituting the above-mentioned human body information generated into a second neural network that has undergone deep learning to perform human body posture and body shape parameter regression, so as to obtain three-dimensional human body parameters, including three-dimensional human body motion posture parameters and three-dimensional human body shape parameters.

[0025] Preferably, in the fitting step, the three-dimensional human body model is made basically consistent with the target human body shape through the three-dimensional human body shape parameters; the three-dimensional clothing model that fits the three-dimensional standard human body model is matched to the three-dimensional human body model through equipment adaptation; the posture of the target human body is determined through the three-dimensional human body action posture parameters, and the skeleton is driven to move from the initial posture to the target posture, so that the three-dimensional human body model is basically consistent with the target human body posture. In the process of equipment adaptation, a field is generated for the mesh patch of the standard human body model, and a field correspondence relationship is established between each patch of the three-dimensional clothing and the corresponding position of the standard human body model. When the body shape of the standard human body model changes, the three-dimensional clothing can also achieve corresponding follow-up changes.

[0026] Preferably, when the model is wearing the garment, the normal fitting state of the three-dimensional garment relative to the standard human body model grid is expanded by a certain size, so that the penetration phenomenon of the three-dimensional garment is reduced.

[0027] Preferably, the fitting driving step further includes the following sub-steps:

[0028] 1) Obtain the position coordinates of the initial posture and the target posture;

[0029] 2) Generate an animation sequence that moves from the initial pose to the target pose;

[0030] 3) In the process of generating animation sequences, grid interpolation is used for processing;

[0031] 4) The interpolation speed is set so that the distance between the initial point and the target point is slow, and the intermediate movement process is fast;

[0032] 5) When driving to the final target posture, stop for several frames to obtain the entire animation sequence;

[0033] 6) Complete the movement of the skeleton from the initial posture to the target posture.

[0034] Preferably, during the movement of the three-dimensional human body model, cloth simulation calculations are performed on the clothes worn by the human body model. The cloth simulation adopts a combination of cloth simulation and skinning methods. The skinning method is used for the part that basically does not deform after the clothes move, and the cloth simulation method is used for the part that deforms during the movement of the clothes. During the cloth simulation process, when the human body model reaches the target posture, several frames of gravity solution are performed on the cloth of the clothes to ensure the realism of the cloth of the clothes in the target posture.

[0035] A virtual fitting system is also provided, comprising: 1) a three-dimensional clothing model making module; 2) a three-dimensional standard human body model making module; 3) a three-dimensional target human body model generating module; 4) a matching module for three-dimensional clothing and three-dimensional target human body models; wherein the three-dimensional clothing making module comprises: 1) a clothing two-dimensional image acquisition module; 2) a three-dimensional standard human body model construction module in combination with a mathematical model; 3) a clothing making module for making a three-dimensional clothing model according to the two-dimensional image of the clothing; 4) a clothing fitting module for fitting the three-dimensional clothing model to the three-dimensional standard human body model; and 5) an inspection module for inspecting the three-dimensional clothing model.

[0036] Preferably, the matching module of the three-dimensional clothing and the three-dimensional target human body model includes: 1) an equipment adaptation module, which is used to adapt the three-dimensional clothing model to the human body model of the target body shape by means of clothing adaptive matching according to the clothing model adapted to the standard human body model and the human body model with a given target posture; 2) a human body model fitting module, which is used to input the obtained several groups of POSE bases and body shape SHAPE three-dimensional human body parameters into the three-dimensional standard human body model for fitting, and use the skeleton to drive the human body model to complete the change from the initial posture to the target posture; 3) a cloth simulation module, which is used to process the cloth so that the clothing maintains authenticity and restoration after following the skeleton movement to the target posture.

[0037] A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the method steps described above is implemented.

[0038] An electronic device, characterized in that it includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement any of the method steps described above when executing the program stored in the memory.

[0039] The beneficial effects of the present invention are:

[0040] 1. Simple user operation. The present invention provides a method for obtaining accurate human body three-dimensional model parameters by analyzing full-body photos of the human body through a deep neural network. Only an ordinary photo is needed to quickly model the human body; at the same time, the three-dimensional clothing model is obtained by processing the two-dimensional clothing pictures in advance. The user does not need to participate in these behind-the-scenes work, but only needs to select the clothing style that he wants to try on virtually, and the system will automatically match the corresponding clothing model. It adapts very well to the characteristics and trends of the Internet era, one is simple, and the other is fast. Users do not need to make any preparations, and uploading a photo is all the work they need to do. If the present invention is applied to entertainment applets or online shopping and other scenarios, it will greatly enhance the user experience and stickiness. Without the need for a depth of field camera or multiple groups of cameras, the 3D model that can be obtained corresponds to the real form of the human body, providing a wide range of application scenarios for various industries, such as clothing and health.

[0041] 2. The virtual clothing has high realism. Here, the realism actually includes two aspects: one is that the clothing naturally follows the changes in the human body state with high realism; the other is that the clothing fabric texture has high restoration. We first put the 3D clothing model on the 3D standard human body model, input the body parameters of the target human body model to obtain the target human body model, and make the 3D clothing model change with the changes in the body shape from the standard human body model to the target human body model through clothing adaptation, and the change of clothing state is very real. In addition, we also use cloth simulation, which is actually to simulate the cloth effect close to the real (of course, it can also be made inconsistent with the physical effect in reality). What we mainly highlight here is the high restoration of cloth texture simulation, including the correctness of cloth printing simulation. Cloth simulation adopts a combination of cloth simulation and skinning method. The part that basically does not deform after the clothes move adopts the skinning method, and the part that deforms during the movement of the clothes adopts the cloth simulation method to ensure the realism and solution speed. During the cloth simulation process, when the human body model reaches the target posture, the cloth of the clothing is subjected to gravity solution for several frames to ensure the realism of the clothing cloth under the target posture.

[0042] 3. Fewer defects in clothing penetration. During the equipment adaptation process, the mesh patches of the standard human body model generate a field, and each patch of the three-dimensional clothing establishes a fixed field correspondence with the corresponding position of the standard human body model. When the standard human body model changes toward the target human body model, the three-dimensional clothing model can also achieve a corresponding uniform change. In the process of generating the corresponding relationship, we will also expand the three-dimensional clothing to a certain size relative to the normal state, which has little effect on the realism of the clothing, but greatly reduces the penetration of the three-dimensional clothing model. It is equivalent to leaving a movable space between the clothing model patch and the human body model mesh, leaving a margin, which will greatly reduce the probability of penetration during the adaptation process of the clothing and the model.

[0043] 4. Add skin brush weight step. After the three-dimensional clothing model is generated, we add a special skin brush weight step, which can actually be called the adjustment of the human body model and the clothing model. It includes two different skin brush weight processing steps between the three-dimensional clothing model and the skeleton, and between the human body model skin mesh and the skeleton. In the prior art, the skin brush weight operation between the human body model skin mesh and the skeleton in the above two operations is usually used alone. However, we creatively combine the two skin brush weight operations. Through this combined operation, the linkage between the skeleton and the clothing is more accurate under the condition of virtual fitting, and the clothing and human skin mesh controlled by each bone can be more accurate and detailed, so as to ensure that there is an improved precise motion matching relationship between the clothing, the human skin mesh and the skeleton. In terms of clothing, the degree of restoration and authenticity are good, and the state of following the movement is basically consistent with the movement in the real world. There will be no defect that the skin mesh and clothing of large blocks or other unrelated areas move abnormally when moving a bone, which can make the following movement of clothing more natural and realistic.

[0044] 5. We designed several sets of standard mannequins in advance, which include the initial posture bases of several common human postures obtained through statistical data processing. Including several basic postures such as sitting, raising hands, raising legs, and closing hands, the posture richness is much higher than that of a single T-pose standard human body model. Under this design, the 3D standard human body model can be a 3D standard human body model with other predetermined postures except the initial T-pose. In the process of moving the 3D human body model, we first calculate which common posture has the shortest distance to move to the target posture, and then choose to fit the standard human body model and the target human body model from the initial posture base of this common posture, which greatly shortens the calculation time of the system, saves computing power, and can drive the standard human body model to the target posture quickly and well, which is very beneficial when the mobile phone may become the most commonly used terminal for virtual clothing change.

[0045] 6. In order to realistically fit the target human body's posture, we used an optimized pin insertion method to complete the change from the initial posture to the target posture. Compared with the traditional pin insertion method, the skeletal information of our target posture is obtained by model regression prediction. At the same time, an animation sequence moving from the initial posture to the target posture is generated; through linear interpolation, nearest neighbor interpolation and other interpolation methods, a time series of skeletal information from the initial posture to the target posture is formed. In the process of generating the animation sequence, the mesh interpolation method is used for processing; the interpolation speed is set to be slow at the distance between the initial point and the target point, and the intermediate movement process is fast; it is especially important that the model is still for several frames when it reaches the final target posture to obtain the entire animation sequence. This approach is closer to the movement laws of the real physical world than uniform interpolation, and the simulation effect of clothing and human posture is better, and it can reduce a considerable amount of processing time.

[0046] 7. High frequency use of deep neural networks. The present invention makes full use of the advantages of deep learning networks and can restore the posture and shape of the human body with high precision in various complex scenes. Different neural networks are used for different purposes, and neural network models with different input conditions and training methods are used to achieve accurate contour separation of the human body in a complex background, semantic segmentation of the human body, determination of key points and joint points, and eliminate the influence of loose clothing and hairstyle, so as to get as close to the real body shape and shape of the human body as possible. Neural network models are also used in the prior art, but due to different input conditions, input parameters, and training methods, the functions and effects of neural network models vary greatly.

[0047] 8. The setting of the neural network model is more scientific and targeted. Some image processing methods in the prior art are too focused on simply outputting the model, without spending time polishing the details of the model. They only complete the mapping from 2D pictures to 3D body models through the training of massive image data. Although the efficiency is very high, the processing flow is too simple and completely relies on the neural network model to generate a three-dimensional human body model. The consistency and effect of the body proportions and details are not satisfactory, and it is completely unhelpful for subsequent further processing, and may become an obstacle that subsequent programs are difficult to overcome. However, in our previous level of neural network, human body contours, human body semantic segmentation, key points and joints are all used as input items, which can complete the generation of model parameters from multiple angles, and the parameters output by the next level of neural network include two categories, POSE and SHAPE, which can control the action and body shape respectively, and combined with our reference model, the posture and body shape of the human body model can be accurately replicated.

[0048] 9. The human body model is precise and controllable. Currently, the popular human body reconstruction methods based on single images are mainly divided into reconstructing parametric human body models. The most commonly used parametric model is the SMPL model of the Max Planck Institute, which contains two sets of 72 parameters for describing human body posture and human body shape. For the single image reconstruction problem, the two-dimensional joint position is first estimated from the image, and then the SMPL parameters are optimized by minimizing the projection distance between the three-dimensional joint and the two-dimensional plane joint, and then the human body is obtained. However, the SMPL model is mainly deeply learned and trained through a large number of human body model instances. The relationship between body shape and shape basis is an overall association relationship, and it is very difficult to decouple. It is impossible to control the desired body part at will, resulting in the generated model not being able to achieve a high degree of consistency with the real human body posture and body shape. In addition, if it is further applied to the subsequent dressing process, it will also lead to limited ability to express the geometric details of the human body surface, and it is impossible to reconstruct the detailed texture of the human body surface clothing well. However, our human body model is not obtained through training. There is a correspondence between parameters based on mathematical principles. In other words, there is no relationship between our various groups of parameters, and they are independent of each other. Therefore, our model is more explanatory during the transformation process and can better represent the shape changes of a certain part of the body. In layman's terms, people have different body shapes, and the proportions of thighs and calves of many people do not meet certain precise proportions. Our model can control the input parameters to achieve the control and length adjustment of the thighs and calves respectively, so as to achieve the precise determination of the leg proportions.

[0049] 10. More suitable for Asian body types. Human body modeling usually designs some standard human body models, which we call standard human body models or basic mannequins. Through the self-built standard human body model, the human body can be manipulated, that is, the process from the starting pose to the target pose can be manipulated. This part of the work is the basis for the clothing to change with the human body pose after completion. Only when the human body accurately reaches the target pose first can the specific process of the clothing following the human body to reach the target pose be calculated. In this process, we abandoned the use of several basic human body models trained by the Max Planck Institute SMPL model based on European body data, and instead established a set of standard human body models (standard mannequins) that are more suitable for Asian body types. This mannequin can include 170 bones and 20 body base parameters, which greatly enriches the details of the human body model and expresses the details better than the SMPL model. And combined with the characteristics of independent control of each base mentioned above, each part of the mannequin can be independently and accurately controlled and modified according to needs, so that each human body model is more beautiful. In addition, we will also make later manual adjustments to the parts of the mannequin, such as the number of vertices and the number of faces, which is a function that other models represented by the SMPL model cannot accomplish. In addition to the precise adjustment of the model's height, other factors such as body shape, arm length, leg proportions, waist length and waist circumference can also be precisely controlled, making the mannequin more closely fit the user's body shape.

[0050] The present invention selects the body base suitable for the characteristics of Asian body shape by establishing its own standard human body model, and uses the parameters corresponding to 20 body bases and 170 bones to generate a 3D human body model that is closer to the Asian body shape than the Max Planck Institute SMPL human body model, and has better independent operation and controllability. At the same time, we use a series of methods to maintain the authenticity and restoration of the 3D clothing model, and use a variety of creative methods in the matching of clothing models and human body models, which not only maintains the fast processing speed, but also maintains the authenticity of clothing simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0052] Figure 1 is a flowchart of the entire process of an embodiment;

[0053] Figure 2 A processing flow chart of a model parameter acquisition module of an embodiment;

[0054] Figure 3 A flowchart of a human body model fitting process according to an embodiment;

[0055] Figure 4 It is a system schematic diagram of the present invention. DETAILED DESCRIPTION

[0056] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0057] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0058] The method for processing a human body image provided by an embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0059] like Figure 1 As shown, an embodiment of the present invention discloses a virtual fitting method, the method comprising: obtaining a two-dimensional image of clothing; making a three-dimensional model of clothing according to the two-dimensional image of clothing; constructing a three-dimensional standard human body model in combination with a mathematical model, wherein the three-dimensional standard model is in an initial posture; fitting the three-dimensional clothing model to a three-dimensional standard human body model in an initial posture, i.e., a standard basic mannequin; obtaining a two-dimensional image of a target human body; obtaining three-dimensional target human body model parameters by calculation of a neural network model; inputting several groups of obtained three-dimensional human body parameters of posture and body shape into the three-dimensional standard human body model for fitting; obtaining a target human body model having the same posture and body shape as the target human body and wearing the changed clothing.

[0060] The virtual fitting method of the present invention generally includes several steps. First, a three-dimensional clothing model is generated; second, a standard human body model is generated, and the three-dimensional clothing model is put on the standard human body model; third, the parameters of the target posture human body model are obtained; fourth, the body shape and posture of the standard human body model are changed to be consistent with the target human body model, and at the same time, the real changes of the three-dimensional clothing model following the changes of the human body model are simulated.

[0061] The first part is mainly to generate a three-dimensional clothing model. In the prior art of generating a three-dimensional clothing model, there are several different methods. At present, the more traditional way of establishing a three-dimensional clothing model is based on the design and stitching method of two-dimensional clothing pieces. This method requires certain clothing expertise to design the sample. Another relatively novel three-dimensional modeling method is based on hand-drawing, and a simple clothing model can be generated through the line information hand-drawn by the user. There is also a method that, on the basis of obtaining clothing picture information, uses image processing technology and graphic simulation technology in combination to finally generate a virtual three-dimensional clothing model. In the picture, the clothing outline and size are obtained through contour detection and classification, and the key points of the edges and edges are found from the outline through machine learning methods. The stitching information is generated through the corresponding relationship between the key points, and finally the clothing is physically stitched in the three-dimensional space. Simulation, obtain the real effect of clothing worn on the human body. In addition, there are mapping methods, mathematical model simulation methods and other methods. The present invention does not make special restrictions on this part, but the three-dimensional clothing model needs to be matched to the standard human body model. The general requirement is to adapt the clothing model that has been adapted to the standard human body model to the human body model under the target posture through the physical simulation of the cloth, and to ensure the naturalness and rationality of the clothing. Therefore, some basic requirements must usually be met, including but not limited to the following: a. It must fit the initial posture of the standard mannequin perfectly without any penetration; b. The output must be a uniform quadrilateral; c. The UV of the model must be unfolded, flattened, compactly aligned, and the texture must be manually aligned with the UV through the Photoshop tool; d. Vertex merging must have been performed; e. The output model should be uniformly reduced in number, and the total number of faces should not exceed 150,000 faces / set according to the reference standard; f. The material must be adjusted in the mainstream clothing design software, and 10 frames of animation must be solved to observe the fabric effect, so that the expected effect can be achieved and the material parameters can be saved; g. The rendering material must be adjusted in the mainstream design software, and a rendering must be previewed to ensure that the material lambert properties are reasonable.

[0062] The second part is to pre-design and model some basic mannequins according to our human modeling method, and put the three-dimensional clothing model on the standard human model to achieve the effect that is compatible with our subsequent workflow. The main work content is: to construct a three-dimensional standard human model, that is, a basic mannequin, in combination with a mathematical model. The SMPL human model of the Max Planck Institute can avoid surface distortion of the human body during movement, and can accurately depict the morphology of human muscle stretching and contraction. In this method, β and θ are the input parameters, where β represents 10 parameters of the proportion of a person's height, weight, head-to-body ratio, etc., and θ represents 75 parameters representing the overall motion posture of the human body and the relative angles of 24 joints. The β parameter is a ShapeBlendPose parameter, which can control the change of human body shape through 10 incremental templates. Specifically, the change of human body shape controlled by each parameter can be depicted by a dynamic image. By studying the continuous animation of parameter changes, we can clearly see that each continuous change of the parameter controlling the human body shape will cause a chain change in the local or even the whole human body model. In order to reflect the movement of human muscle tissue, the linear change of each parameter of the SMPL human body model will cause a large area of ​​mesh changes. Figuratively speaking, for example, when adjusting the parameters of β1, the model will directly understand the parameter changes of β1 as the changes of the entire body. You may only want to adjust the proportion of the waist, but the model will force you to adjust the fatness and thinness of the legs, chest, and even hands. Although this working mode can greatly simplify the workflow and improve efficiency, it is indeed very inconvenient for projects that pursue modeling effects. Because the SMPL human body model is, after all, a model that conforms to the Western body shape trained through Western human body photos and measurement data. The law of its body shape change basically conforms to the usual change curve of Westerners. When applied to the modeling of Asian human body models, many problems will arise, such as the proportion of arms and legs, the proportion of waist and body, the proportion of neck, the length of legs and arms, etc. Through our research, there are large differences in these aspects. If the SMPL human body model is rigidly applied, the final generation effect cannot meet our requirements.

[0063] To this end, we adopted the method of self-made human body model to improve the effect. The core is to build a human body blendshape base to achieve accurate independent control of the human body. Preferably, the three-dimensional standard human body model (basic human platform) is composed of 20 body base parameters and 170 bone parameters. The several bases constitute the entire human body model, and each body base is controlled by parameters separately, without affecting each other. The so-called precise control, on the one hand, increases the control parameters, and does not follow the ten β control parameters of the Max Planck Institute. In this way, the adjustable parameters are not only the usual fat and thin, but also the length of the arm, the length of the leg, the fat and thin waist, hips and chest, etc., and the parameters are more than doubled in terms of bone parameters, which greatly enriches the range of adjustable parameters and provides a good foundation for the refined design of standard human body models. The so-called independent control can be understood as each base is controlled separately, such as waist, legs, hands, head, etc., and each bone can also be adjusted in length separately, which is independent of each other and will not produce physical linkage, so that the human body model can be finely adjusted. The model no longer looks "dumb and clumsy" and can never be adjusted to a shape that satisfies the designer. Our existing model reflects a correspondence based on mathematical principles. In fact, it is equivalent to designing this model from two parts: artificial aesthetics and data statistical analysis, so that it can generate the correct model that we think is suitable for the Asian body shape according to our design rules, which is significantly different from the big data training model of the SMPL human body model. Therefore, our parameter transformation is more interpretable and can better characterize the local shape changes of the human body model. Moreover, this change is based on mathematical principles. There is no influence between the various parameters, and the arms and legs remain completely independent. In fact, so many different parameters are designed to avoid the defects of the human body model trained by big data, and to accurately control the human body model in more dimensions, not limited to a few indicators such as height, which greatly improves the modeling effect. Only under the premise of self-built body base, setting so many independent control parameters is of practical significance. To meet the requirements of designer level, both are indispensable.

[0064] As for putting the three-dimensional clothing model on the standard human body model, it is a conventional technology in the field, and the present invention does not make too many restrictions on this, as long as the required effect can be achieved.

[0065] The third part is to process the acquired human body images to obtain the parameter information required to generate the human body model. In the past, the selection of these skeletal key points was usually done manually, but this method is very inefficient and does not adapt to the fast-paced requirements of the Internet era. Therefore, in today's era of neural networks, the use of deep learning neural networks instead of manual key point selection has become a trend. However, how to use neural networks efficiently is an issue that requires further research. Generally speaking, we adopted the idea of ​​​​a two-level neural network plus data "fine-tuning" to construct our parameter acquisition system. Figure 2 As shown, we use a deep learning neural network to generate these parameters, which mainly includes the following sub-steps: 1) obtain a two-dimensional image of the target human body; 2) process and obtain a two-dimensional human body contour image of the target human body; 3) substitute the two-dimensional human body contour image into the first deep learning neural network for joint point regression; 4) obtain a joint point map of the target human body; obtain semantic segmentation maps of various parts of the human body; body key points; body bone points; 5) substitute the generated joint point map, semantic segmentation map, body bone points and key point information of the target human body into the second deep learning neural network for human posture and body shape parameter regression; 6) obtain the output three-dimensional human body parameters, including three-dimensional human body action POSE parameters and three-dimensional human body shape SHAPE parameters.

[0066] The two-dimensional image of the target human body can be a two-dimensional image including a human body image in any posture and any clothing. The acquisition of the two-dimensional human body contour image utilizes a target detection algorithm, which is a target region rapid generation network based on a convolutional neural network.

[0067] Before the two-dimensional human image is input into the first neural network model, the process of training the neural network is also included. The training sample includes a standard two-dimensional human image with the original joint point position marked, and the original joint point position is manually marked on the two-dimensional human image with high accuracy. Here, the target image is first obtained, and the target image is detected by the target detection algorithm. Human body detection does not use a measuring instrument to detect the real human body. In the present invention, it actually refers to any given image, which is usually a two-dimensional photo containing enough information, such as a face, and the limbs and body of the person are required to be included in the picture. Then, a certain strategy is used to search the given image to determine whether the given image contains a human body. If the given image contains a human body, the position, size and other parameters of the human body are given. In this embodiment, before obtaining the key points of the human body in the target image, it is necessary to perform human body detection on the target image to obtain a human body frame with the human body position marked in the target image. Because the picture we input can be any picture, it is inevitable that there are some backgrounds of non-human images, such as tables and chairs, big trees, cars, buildings, etc., and these useless backgrounds must be removed through some mature algorithms.

[0068] At the same time, we also need to perform semantic segmentation, joint point detection, bone detection and edge detection. After collecting these 1D point information and 2D surface information, we can lay a good foundation for generating a 3D human body model later. Use the first-level neural network to generate a human joint point map. Optionally, the target detection algorithm can quickly generate a network for the target area based on the convolutional neural network. The first neural network needs to be trained with a large amount of data. Some photos collected from the network are manually annotated with joint points, and then input into the neural network for training. After deep learning of the neural network, it is basically possible to immediately obtain a joint point map with the same accuracy and effect as the manually annotated joint points after inputting the photo, and the efficiency is dozens or even hundreds of times that of manual annotation. Human joints usually exist as human key points. Human key points include important key points. Important key points include, for example, top of the head key points, neck key points, shoulder key points, elbow key points, wrist key points, hand key points, armpit key points, waist key points, hip key points, knee key points, ankle key points, toe top key points and two legs center key points. Optionally, the key points of the human body include 1 key point on the top of the head, 2 key points on the neck, 2 key points on the shoulders, 4 key points on the elbows, 4 key points on the wrists, 2 key points on the hands, 2 key points on the underarms, 2 key points on the waist, 2 key points on the hips, 4 key points on the knees, 4 key points on the ankles, 2 key points on the tops of the toes and 1 key point on the center of the legs. Optionally, the number of key points of the human body is 68. For key points other than the important key points in the key points of the human body, they can be determined according to the positions of the important key points. For example, the bisection point of the plane distance between two important key points in a two-dimensional plane can be used as a key point.

[0069] In the present invention, obtaining the joint position of the human body in the photo is only the first step. After obtaining the 1D point information, the 2D surface information needs to be generated based on the 1D point information. These tasks can be completed by the neural network model and the mature algorithms in the prior art. The present invention redesigns the process and intervention timing of the neural network model, and reasonably designs various conditions and parameters, so that the parameter generation work is more efficient and the degree of manual participation is reduced. It is very suitable for Internet application scenarios. For example, in a virtual dress-up program, users do not need to wait and can basically get the dress-up result instantly, which plays a vital role in increasing the attractiveness of the program to users.

[0070] After obtaining the relevant 1D point information and 2D surface information, these parameters or results, the joint point map of the target human body, the semantic segmentation map, the body bone points and the key point information can be substituted as input items into the second neural network after deep learning to regress the human body posture and body shape parameters. After the regression calculation of the second neural network, several groups of three-dimensional human body parameters can be immediately output, including three-dimensional human body action POSE parameters and three-dimensional human body shape SHAPE parameters. Preferably, the loss function of the neural network is designed according to the three-dimensional standard human body model (basic mannequin), the predicted three-dimensional human body model, the standard two-dimensional human body image with the original joint point positions marked, and the standard two-dimensional human body image including the predicted joint point positions.

[0071] The fourth part, which is also the most critical part, is to fit the parameters of the human body model with the human body model, and at the same time, ensure that the state of the clothes after moving is as realistic as possible.

[0072] like Figure 3 As shown, the moving process includes the following sub-steps: the obtained three-dimensional human body POSE and SHAPE parameters are matched with several bases and bone parameters of the three-dimensional standard human body model; the obtained several groups of bases and bone parameters are input into the standard three-dimensional human body parameter model for fitting; the three-dimensional human body model has a mathematical weight relationship between the bone points and the model grid, and the determination of the bone points can be associated with the human body model of the target human body posture.

[0073] In this part, the two parameters generated in the previous part can be used to substitute the pre-designed human body model to construct a 3D human body model. These two types of parameters are similar in name to the parameters of the human body SMPL model of the Max Planck Institute, but the actual substantive contents they contain are quite different. Because the basis of the two is not the same, that is to say, the present invention adopts a self-made three-dimensional standard human body model (basic mannequin), each of which is designed by ourselves according to the body shape and body proportion of Asians, including several parts that the SMPL model does not involve, and the SMPL model of the Max Planck Institute adopts a standard human body model generated by big data training. The generation and calculation methods of the two models are different. Although they are finally embodied as the generated 3D human body model, their connotations are quite different. After this step, a preliminary 3D human body model will be obtained, including the mesh of the human body model of the skeleton position and length information.

[0074] In this part, after putting the 3D clothing model on the standard human body model, we need to make the body shape and posture of the standard human body model consistent with the target human body, and simulate the real changes of the 3D clothing model following the changes of the human body model. We use several methods to ensure the smooth achievement of the above goals.

[0075] First of all, in addition to the initial T-pose, the 3D standard human model can also be a 3D standard human model with other predetermined poses. We will design multiple sets of standard mannequins in advance, which include the initial pose bases of several common human poses obtained through statistical data processing, including A-POSE, sitting, hand raised, leg raised, hand crossed and folded in front of the chest, arms hanging naturally, sideways and other common basic poses. The richness of poses is much higher than that of a single T-pose standard human model. Under this design, the 3D standard human model can be a 3D standard human model with other predetermined poses except the initial T-pose. In the process of moving the 3D human model, first calculate which common pose's initial pose base has the shortest comprehensive distance to move to the target pose, and then choose to fit the standard human model and the target human model from the initial pose base of this common pose to realize the movement of the human model, which greatly shortens the calculation time of the system, saves computing power, and can drive the standard human model to the target pose quickly and well, which is very beneficial in the case that the mobile phone may become the most commonly used terminal for virtual clothing change.

[0076] Secondly, after the three-dimensional clothing model and human body model are generated, we add a special skin brush weight step, which can actually be called the adjustment of the human body model and clothing model. It includes two different skin brush weight processing steps between the three-dimensional clothing model and the skeleton, and between the human body model skin mesh and the skeleton. In the prior art, the skin brush weight operation between the human body model skin mesh and the skeleton is usually used alone in the above two operations. However, we creatively combine the two skin brush weight operations. Through this combined operation, the linkage between the skeleton and the clothing is more accurate under the condition of virtual fitting, and the clothing and human skin mesh controlled by each bone can be more accurate and detailed, so as to ensure that there is an improved precise motion matching relationship between the clothing, the human skin mesh and the skeleton. In terms of clothing, the restoration and authenticity are good, and the state of following the movement is basically consistent with the movement in the real world. There will be no defect that the skin mesh and clothing of large blocks or other unrelated areas move abnormally when moving a bone, which can make the following movement of clothing more natural and realistic.

[0077] In actual operation, in 3D modeling and animation software, select the skeleton and model, skin-bind skin, and the model will be displayed in different colors after binding. The skin details that are automatically brushed on are often not good, so we need to adjust them separately. Then open the skin weight drawing tool, you can see the information of each joint and node. After selecting one of the joints, the corresponding control area can be displayed, which clearly shows the parent-child relationship between the upper and lower levels. The upper node will control the lower node. This is why some joints do not control a certain area to affect the movement of the part, because the child nodes it controls have an impact on it, and the impact of the child nodes is transmitted by it; you can see that the position controlled by the joint has turned white. Usually the joints are hidden in the skin mesh and it is not convenient to adjust directly, so the controller is used to control the joints, and how the joints control the model is achieved through skinning, that is, affecting the mesh patches of the model, and the size of the impact is achieved through weights, that is, the degree to which each patch is affected. If a more realistic effect is required, the weights need to be brushed to make the model change according to our understanding of the real state. Usually white represents 1 and black represents 0. The more control, the greater the impact, the closer the weight is to 1, and the whiter the color. By brushing weights for the human body model mesh and clothing model mesh separately, the unnaturalness of automatic weight brushing can be avoided, making the human body mesh and clothing mesh more consistent with their actual state, and reflecting a more realistic following effect during movement.

[0078] Again, the so-called equipment adaptation means that, based on the clothing model that has been adapted to the standard human body model, given a target posture human body model, that is, a human body model with the same posture and mesh structure as the standard human body model, the same number of mesh vertices and mesh units, and the same topological connection relationship between vertices, but with different body shapes such as height, weight, etc., the three-dimensional clothing model is adapted to the human body model of the target body shape through clothing adaptive matching.

[0079] Because in the real world, people vary greatly in height, weight, and appearance, and different people will look very different when wearing the same piece of clothing. Therefore, if we want to ensure the realism of the target human model and clothing model, we must also allow the clothing model to change with the real human body shape and posture. On the technical level, as the human body parameters increase or decrease, the standard human body model mesh will move, and in the adaptive state, the movement of the human body mesh will drive the clothing model to move together, and there is a definite binding and linkage relationship between the mesh and the clothing. In our solution, the driven bodies are all driven by the driving source mesh and move together.

[0080] The most basic and initial use of equipment adaptation in the animation field is to solve the problem of body shape. Under a fixed initial state Tpose, when the human model becomes fatter or thinner, the clothes must also become fatter or thinner. In the initial state, the clothing model fits on the grid of the standard human model. As the grid expands or contracts, the clothes will also show a trend and state of following. In fact, it is equivalent to using an adaptive algorithm to simulate the changes in body shape of real clothes worn on real people, so that the authenticity after the change of clothes can be guaranteed.

[0081] The target human body's posture is determined by the 3D human body motion posture parameters, and the human body model mesh and clothing model are driven to move from the initial posture to the target posture in an adaptive manner, so that the 3D human body model is basically consistent with the target human body posture, and the clothing model is matched with the target human body posture.

[0082] In the field of animation, it is common to use bones or joints as the source of drive and skin drive. The so-called skin drive is to establish the relationship between the model mesh vertices and bones during the driving process. After calculating the rotation parameters of the bones at the target pose, the bones can drive the vertices near them to move in space through the relationship of skin weights, and drive the associated vertices to the position of the target pose. For example, when the bones are bent and rotated, the influence drives the nearby bound mesh vertices to move in space. In the same way, due to the existence of our self-built human body model, there are many parameters representing human body characteristics, which can express every small part of the human body model in a relatively fine manner. For example, after we obtain the rotation parameters of all the target poses (pose) of 170 bones, we can use all the skin weights to drive the mesh vertices of the entire human body model to the target pose (pose). The advantage of skin drive is that it is very fast. It can be understood that it is not necessary to insert frame calculations for each frame, and the clothing model can be driven directly from the initial state to the target pose. However, its disadvantages are also obvious. There is no physical world simulation operation content, so the change process is very unnatural, and the state after reaching the target position is different from the state after the actual movement. For tight clothing models, there is more penetration, and for loose 3D clothing models, it can be clearly seen that it is unnatural. This will cause the process to be rough and the penetration phenomenon to be more serious. There is no big defect in using it in the game, but in the field of virtual dressing, this method cannot achieve the best effect in terms of realism and accuracy.

[0083] In addition, skinning can directly obtain the mesh position of the target pose from the initial Tpose through the skinning weight matrix. However, the field relationship based on equipment adaptation to drive the clothing model cannot be directly obtained through matrix multiplication, because this field will fail when the span between the initial pose and the target pose is relatively large. Therefore, the initial use of equipment adaptation can only be used in situations where the body shape of the human model changes under a fixed Tpose, and the clothing model also changes accordingly, that is, in a static state, the height, weight, and size of the clothing can be changed according to the different body shapes of the human model.

[0084] However, after our research, we found that the vertex relationship field of the equipment adaptive module is highly robust and can fully support small-scale action differences. Therefore, we creatively use the accumulation of small-scale actions to form large-scale actions, and use the equipment adaptive module to drive the clothing model frame by frame to reach the target pose. At the same time, based on the characteristics of the equipment adaptive driving method, we avoid using a one-step driving method. Our adaptive algorithm is different from the binding between the skinned human mesh and the skeleton. Instead, it establishes a relationship between the human mesh and the clothing mesh, emphasizing the concept of field. Through the interpolation between the initial pose and the target pose, the model mesh is driven frame by frame to reach the target pose. By calculating the mesh frame by frame, the matching result of the final target human model and the clothing model is obtained. Each driven model vertex is acted upon by multiple driven meshes around it, and the force of the action is relatively even. This method rarely penetrates the model during the driving process, achieving a more realistic simulation effect.

[0085] For example, the realistic simulation of the natural drooping state of loose clothing mentioned earlier is more difficult for skinning operations, but the adaptive equipment method can solve similar problems very well, because it is a frame-by-frame movement calculation, which will form a collection of small movements, and there will be no large-scale one-step movement of the clothing model, which greatly reduces the possibility of clothing models penetrating the model. In practice, we can already test single-layer clothing with a clothing mesh of 100,000 triangles (or 50,000 quads) as the benchmark, and require the same effect. When using the clothing adaptive module, it takes less than 1 second, and at least 90% of the clothing set models do not penetrate the model.

[0086] In the process of equipment adaptation, it is equivalent to generating a field on the mesh patches of the standard human body model. Each patch of the three-dimensional clothing establishes a field correspondence with the mesh patches at the corresponding positions of the standard human body model. When the mesh of the standard human body model changes, the three-dimensional clothing can also achieve corresponding follow-up changes.

[0087] The equipment adaptation process is achieved by interpolating frames between the initial posture and the target posture, and then driving the human model mesh frame by frame in small amplitude. First, a specific connected mesh is designated as the driving source, and the specific connected mesh is designated as the driven mesh. The method according to claim 8 is characterized in that a relationship field is established according to the driving relationship between the driving source and the driven mesh, so that at the initial position, for a vertex coordinate of the human model mesh as the driving source, it is represented by (x, y, z), and the mesh vertices of the driven model with the maximum number of constraints of 10 closest to it in the three-dimensional space are calculated, and a vertex-vertex relationship matrix, that is, a vertex relationship field, is established. When the human body shape changes, the shape of the driving source of the human model changes. According to the changed driving source mesh vertices, the vertex relationship field matrix is ​​multiplied to obtain the new three-dimensional space coordinates of the changed clothing model, and the equipment adaptation process is completed.

[0088] Again, in the process of generating the corresponding relationship, we will also expand the three-dimensional clothing to a certain size relative to the normal state, which has little effect on the realism of the clothing, but greatly reduces the penetration phenomenon of the three-dimensional clothing model. The expansion method refers to the establishment of a distance correspondence between each face of the three-dimensional clothing and the corresponding grid position of the standard human body model, which is equivalent to leaving a space for movement between the clothing model face and the human body model grid, leaving a margin, which will greatly reduce the probability of penetration in the process of clothing and model adaptation. Of course, in the specific corresponding process, different parameters can be selected for correspondence, such as grid vertices. There will be a situation where it is not a point-to-point correspondence. Sometimes a point may correspond to an area or block, which is also a corresponding relationship. When the human body model changes, the three-dimensional clothing can also achieve corresponding follow-up changes. For example, when the model is driven from the initial posture to the target posture, our clothing model will also move with the model. In the process of generating the corresponding relationship, we will expand the 3D clothing to a certain size relative to the normal fit state, so that the 3D clothing has additional redundancy, which is very helpful for the integrity of the human body model after a large range of large movements. Due to the excessive movement of the limbs, the clothing model will cause many problems of penetration between the clothing model and the limbs due to the untimely and inaccurate following of the movement and the collision of the limbs caused by the movement. This is actually the most easily perceived major defect in the field of virtual fitting. To address this problem, we first proposed a solution to expand the clothing model relative to the human body model by a space, hoping that this extra isolation space can play a role in maintaining the integrity of the fit between the clothing model and the human body model. In the past, technicians usually pursued a close fit between the clothing model and the human body model, believing that this can more realistically reflect the effect of the clothes on the body. However, in practice, the tighter the two are, the greater the possibility of problems. We are equivalent to doing the opposite, solving this problem from the opposite idea, and artificially adding a buffer space between the clothing model and the human body model. Here, our expansion is to translate a certain absolute spatial distance outward along the normal direction of the human body mesh. For example, at the elbow of the human body, the vertices of the clothing model are moved outward by 1 cm along the normal direction of the human body mesh. Then the clothes at the elbow have a more redundant distance from the human body model, and there will be greater redundancy in the process of cloth solving or skinning driving, which greatly reduces the proportion of penetration. Of course, our expansion still follows certain rules. For example, in the armpit area, along the normal direction of the human body model mesh, each vertex of the clothing model is moved outward by 1 cm. There is a high probability that clothes will penetrate each other. The penetration at this position needs to be repaired by the penetration repair module. In practice, the outward expansion distance is usually set differently for each part according to certain preset rules.For example, the arms and legs can be extended outwards appropriately, but the armpits and the outside of the hips are generally not extended. From the current implementation, the effect of preventing penetration is very good. At the same time, there is no visible impact on the fit and aesthetics of the human body model and the clothing model. After a certain number of experiments, we found that this improvement has little effect on the realism of the clothing, but it greatly reduces the penetration of the three-dimensional clothing model. It is equivalent to leaving a space between the clothing model patch and the human body model mesh to accommodate the abnormal movement of the human body model and clothing model, leaving a margin, which will greatly reduce the probability of penetration during the adaptation process of the clothing and model.

[0089] Again, in this part, after putting the 3D clothing model on the standard human body model, it is necessary to make the body shape and posture of the standard human body model consistent with the target human body, and at the same time simulate the real changes of the 3D clothing model following the changes of the human body model.

[0090] The target human body posture is determined by the three-dimensional human body action posture parameters, and the skeleton is driven to move from the initial posture to the target posture, so that the three-dimensional human body model is basically consistent with the target human body posture. In the field of animation, it is common to use bones or joints as the driving source and use skinning driving. The so-called skinning driving is to establish the relationship between the model mesh vertices and bones during the driving process. After calculating the rotation parameters of the bones at the target posture, the bones can drive the vertices near them to complete the spatial movement through the relationship of the skinning weights, and drive the associated vertices to the position of the target posture. For example, when the bones are bent and rotated, the influence drives the nearby bound mesh vertices to complete the spatial movement. In the same way, due to the existence of our self-built human body model, there are many parameters representing human body characteristics, which can express every small part of the human body model in a relatively fine manner. For example, after we obtain the rotation parameters of all the target poses of 170 bones, we can use all the skinning weights to drive the mesh vertices of the entire human body model to the target posture. The advantage of skinning driving is that it is very fast, which can be understood as not needing to insert frame calculations for each frame, and can directly drive the clothing model to the target posture from the initial state. However, its shortcomings are also obvious. There is no physical world simulation operation content, so the change process is very unnatural, and the state after reaching the target position is different from the state after the actual movement. For tight clothing models, the penetration is not obvious, but for loose 3D clothing models, it is obvious that it is unnatural.

[0091] To address these issues, we use a combination of cloth simulation and skinning methods to move clothing models. Skinning is used for parts that are basically not deformed after the clothing moves, and cloth simulation is used for parts that are deformed during the clothing movement. Here, the skinning method is mainly used to improve the processing speed of movement, and the cloth simulation method is mainly used to improve the realism. The so-called cloth simulation is actually to simulate a cloth effect that is close to reality (of course, it can also be made inconsistent with the physical effect in reality). Here we mainly highlight the high restoration of cloth texture simulation, including the accuracy of cloth printing simulation. Our method takes into account both texture and speed. On the premise of ensuring that the cloth simulation effect exceeds that of general models, in some parts that are not particularly important and the deformation is not very drastic, the skeleton-driven skinning grid is used to drive the model to move. The driving of the clothing model adopts a combination of cloth simulation and skinning methods. The skinning method is used to ensure the solution speed for parts that are basically not deformed after the clothing moves, such as the upper body. The cloth simulation method is used to ensure the realism and cloth texture of the parts that are deformed during the movement of the clothing, such as the legs and the hem of the clothing.

[0092] The movement state from the initial posture to the target posture is calculated frame by frame. When calculating each frame, the skin drive is completed first and then the cloth simulation is calculated. After the current frame is calculated, the state of the next frame is calculated. This actually forms a sequence of interpolation states, maintaining the high authenticity brought by frame-by-frame movement and frame-by-frame calculation. Cloth simulation uses a collision body system to model the vertex mesh of the human model as a rigid body and the mesh of the clothing model as a non-rigid body. The physical engine is used to simulate the collision relationship between rigid bodies and non-rigid bodies. The collision between rigid bodies and non-rigid bodies is calculated during the cloth simulation process. At the same time, the connection force between the clothing model's own mesh is considered, and the mesh state of the clothing model is calculated frame by frame to simulate the collision movement process in the physical world. This frame-by-frame simulation method can maintain the real state of the clothing model's movement to the greatest extent. Although the simulation speed has been reduced, it has also maintained a relatively good balance in the context of enhanced device computing power.

[0093] Again, the driving method of the present invention has another important characteristic. When driving the posture of the human body model to reach the target posture, it is necessary to perform several frames of gravity solution on the fabric of the clothing, judge the continuity of the action change according to the neural network prediction model, and analyze whether the target posture is a snapshot or a static pose. If it is a snapshot, the clothing model will maintain the speed during the movement due to inertia, making the target posture state a non-steady state. When the human body model reaches the target posture, the number of gravity solutions is reduced; if it is a pose, the number of gravity solutions is increased when the human body model reaches the target posture to ensure the fidelity of the clothing fabric in the target posture. Especially at the end of the movement, several frames of fabric solution are performed on the clothing, which perfectly matches the variable speed interpolation drive in time. This process originally exists, and it is originally for obtaining a stable model state. At the same time, it is applied to the clothing model, using a parallel method, and there is basically no loss in time. The final texture state of the clothing is reproduced more completely, and the authenticity is improved. During the fabric simulation process, when the human body model reaches the target posture, a neural network is used to determine the static or moving state of the input image. According to different situations, the gravity solution of the clothing fabric is performed for several frames to simulate the final state of the clothes under the real motion state, so as to ensure that the clothing fabric presents a movement trend consistent with the motion form of the photo under the target posture, achieving good realism and texture.

[0094] Finally, in this part, the human body model must complete the change from the initial posture to the target posture. Because we only input a photo, the target human body posture in the photo is usually different from the basic mannequin. At this time, in order to fit the target human body posture, it is necessary to complete the change from the initial posture to the target posture. In order to simulate more realistically in the fitting step, when several groups of basis and bone parameters are fitted in the standard 3D human body parameter model, the following steps are also included:

[0095] 1) Obtain the position coordinates of the initial posture and the target posture; the acquisition of the initial posture parameters is determined by the initialization parameters of the standard mannequin model, and the skeleton information of the target posture is obtained by regression prediction of the neural network model.

[0096] 2) Generate an animation sequence that moves from the initial posture to the target posture; after obtaining the initial state of the skeleton information and the state parameters of the target posture, form a time series of skeleton information from the initial posture to the target posture through linear interpolation, nearest neighbor interpolation and other interpolation methods. In the driving process, according to the number of bones driven per frame, it can be divided into two methods: global linear interpolation and driving the parent node first and then the child node. Considering the driving state in the simulated physical world, this patent adopts the latter method, driving the parent skeleton node first and then the child skeleton node. In this way, the interpolated animation sequence action is more in line with the real physical world, and the simulation effect is better.

[0097] 3) In the process of generating animation sequences, mesh interpolation is used for processing; that is, after each frame drives the skeleton to move, the current state of the human body model vertex or face information is calculated through the weight parameters of the standard mannequin, and the current human body model mesh state is updated and recorded and saved.

[0098] 4) The interpolation speed is set to be slow in the distance between the initial point and the target point, and fast in the middle of the movement; this patent adopts a non-uniform interpolation speed, that is, the single frame movement amplitude is small during the initial action and the end of the action, and the movement amplitude is large during the movement. The starting state of the physical action in the real world has a certain acceleration process, and the displacement distance between frames is maintained at a high level during the movement, and the driving speed is reduced at the end of the movement.

[0099] 5) When driving to the final target posture, stop for a few frames to get the entire animation sequence. This approach is closer to the motion law of the real physical world than uniform frame insertion, and the simulation effect is better.

[0100] 6) Complete the movement of the skeleton from the initial posture to the target posture.

[0101] Since we have obtained the skeleton data and mesh information and data, it will be easier to drive the skeleton in this case. We can use the LBS algorithm and DQS algorithm. The collision body should also be considered. Because the model of the standard mannequin is in a three-dimensional standard posture, the change from the initial posture to the target posture may cause unreasonable interpenetration between the meshes of the human body model. Only by combining the collision body can we ensure that there is no interpenetration defect between the meshes.

[0102] In addition, the present invention also includes the step of refining the obtained human body model, including the following sub-steps: obtaining a three-dimensional human body model with the same posture and body shape as the target human body; obtaining a final three-dimensional target human body model grid that accurately fits the two-dimensional image of the target human body.

[0103] In this part, we get the fitted 3D human body model. If the target human body posture is complex, accurate fitting can be achieved through multiple fittings. However, this fitting is completed in multiple levels, not an average value. Instead, it adopts a multi-level iteration method. The first level quickly drives the skeleton to the vicinity of the target posture to ensure high efficiency in speed, and the second level makes fine adjustments according to the algorithm. This can ensure a balance between the fitting speed and the final modeling quality.

[0104] The virtual fitting method of the present invention can greatly reduce the amount of calculation and complexity when generating a three-dimensional human body model through advance deep learning and simulated motion settings, and can achieve an effect that far exceeds the realism of human body models generated by existing 2D pictures. At the same time, through the equipment adaptive method, the pre-made three-dimensional clothing model is naturally and realistically worn on the target human body model, which well simulates the image of clothing worn on the human body.

[0105] Combination Figures 1 to 3 The method for generating a three-dimensional human body model according to the embodiment of the present invention may be implemented by a human body image processing device. Figure 4 FIG. 3 is a schematic diagram showing a hardware structure 300 of a device for processing a human body image according to an embodiment of the present invention.

[0106] The present invention also discloses a virtual fitting system, comprising: 1) a 3D clothing model making module; 2) a 3D standard human body model making module; 3) a 3D target human body model generating module; 4) a 3D clothing and 3D target human body model matching module;

[0107] Furthermore, the three-dimensional clothing production module includes: 1) a clothing two-dimensional image acquisition module; 2) a three-dimensional standard human body model construction module combined with a mathematical model; 3) a clothing production module for producing a clothing three-dimensional model according to the two-dimensional image of the clothing; 4) a clothing fitting module for fitting the three-dimensional clothing model to the three-dimensional standard human body model; 5) an inspection module for inspecting the clothing three-dimensional model.

[0108] Furthermore, the matching module of the three-dimensional clothing and the three-dimensional target human body model includes: 1) an equipment adaptive module, which is used to adapt the three-dimensional clothing model to the human body model of the target body shape by means of clothing adaptive matching according to the clothing model that has been adapted to the standard human body model and the human body model with a given target posture; 2) a human body model fitting module, which is used to input the obtained several groups of POSE bases and body shape SHAPE three-dimensional human body parameters into the three-dimensional standard human body model for fitting, and use the skeleton to drive the human body model to complete the change from the initial posture to the target posture; 3) a cloth simulation module, which is used to process the cloth so that the clothing maintains authenticity and restoration after following the skeleton movement to the target posture.

[0109] Furthermore, the three-dimensional target human body model generation module includes: 1) a two-dimensional human body image acquisition module; 2) a joint point map acquisition module; 3) a semantic segmentation map acquisition module; 4) a human body posture and body shape parameter regression module; 5) a standard three-dimensional human body parameter model fitting module; 6) a three-dimensional human body model data processing and refinement module; 7) a target human body model mesh output module.

[0110] And a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the virtual clothing changing method and steps described above are implemented.

[0111] And an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement the virtual clothing changing method and steps described above when executing the program stored in the memory.

[0112] like Figure 4 As shown, the device 300 for implementing virtual fitting in this embodiment includes: a processor 301, a memory 302, a communication interface 303 and a bus 310, wherein the processor 301, the memory 302, and the communication interface 303 are connected through the bus 310 and communicate with each other.

[0113] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0114] Memory 302 may include a large capacity memory for data or instructions. For example, but not limitation, memory 302 may include HDD, floppy disk drive, flash memory, CD, magneto-optical disk, tape or universal serial bus (USB) driver or two or more of these combinations. In appropriate cases, memory 302 may include removable or non-removable (or fixed) media. In appropriate cases, memory 302 may be inside or outside of processing human body image device 300. In a specific embodiment, memory 302 is a non-volatile solid-state memory. In a specific embodiment, memory 302 includes a read-only memory (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or two or more of these combinations.

[0115] The communication interface 303 is mainly used to implement the communication between the modules, devices, units and / or equipment in the embodiment of the present invention.

[0116] Bus 310 includes hardware, software or both, and the parts of the equipment 300 for processing human body image are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In suitable cases, bus 310 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0117] That is to say, Figure 4 The device 300 for processing human body images shown in the figure can be implemented as including: a processor 301, a memory 302, a communication interface 303 and a bus 310. The processor 301, the memory 302 and the communication interface 303 are connected through the bus 310 and communicate with each other. The memory 302 is used to store program codes; the processor 301 reads the executable program codes stored in the memory 302 to run the program corresponding to the executable program codes, so as to execute the virtual fitting method in any embodiment of the present invention, thereby realizing the combination of Figures 1 to 3 The invention describes a method and apparatus for virtual fitting.

[0118] The embodiment of the present invention further provides a computer storage medium, on which computer program instructions are stored; when the computer program instructions are executed by a processor, the method for processing a human body image provided by the embodiment of the present invention is implemented.

[0119] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0120] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0121] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0122] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A virtual fitting method, the method comprising the following steps: S1: Obtain a two-dimensional image of clothing; S2: creating a three-dimensional clothing model of the clothing according to the two-dimensional image of the clothing; S3: constructing a three-dimensional standard human body model in combination with the mathematical model, wherein the three-dimensional standard human body model is in an initial posture; In step S3, the three-dimensional standard human body model is a three-dimensional standard human body model in other predetermined postures except the initial T-pose; several initial posture bases of common postures are pre-designed, and in the process of moving the three-dimensional human body model, the distance from which common posture to the target posture is the shortest is calculated first, and then the initial posture base of the common posture is selected to start fitting, so as to improve the processing speed; The three-dimensional standard human body model is composed of parameters of several body bases and several root bone parameters, wherein the several bases constitute the entire human body model, and each body base is individually controlled by parameters; The control parameters have been increased. The ten B control parameters of the Max Planck Institute are not used. The adjustable parameters include fatness, arm length, leg length, waist, hips and chest fatness. The parameters of bone parameters have been increased by more than double, which provides a good foundation for the refined design of standard human body models. Each base is controlled individually, and the length of each bone can be adjusted individually. They are independent of each other and will not produce any physical linkage. The three-dimensional human body model has a mathematical weight relationship between skeleton points and human body mesh, and the determination of the skeleton points can be associated with the human body model of the target human body posture; S4: fitting the three-dimensional clothing model to a three-dimensional standard human body model in an initial posture, comprising: The 3D human body model is made basically consistent with the target human body shape through the 3D human body shape parameters; the 3D clothing model that fits the 3D standard human body model is matched to the 3D human body model through equipment adaptive mode; Determine the target human body's posture through the 3D human body motion posture parameters, and drive the skeleton to move from the initial posture to the target posture, so that the 3D human body model is basically consistent with the target human body posture; In the process of equipment adaptation, a field is generated for the mesh patch of the standard human body model, and a corresponding relationship is established between each patch of the three-dimensional clothing and the corresponding position of the standard human body model. When the body shape of the standard human body model changes, the three-dimensional clothing model can also achieve corresponding follow-up changes. S5: Acquire a two-dimensional human body contour image of the target human body; S6: Obtaining the three-dimensional target human body model parameters through the calculation of the neural network model, including: S61: Processing the obtained two-dimensional human body contour image of the target human body; S62: Substituting the two-dimensional human body contour image into a first neural network that has undergone deep learning to perform joint point regression; S63: Obtain a joint point map of the target human body and a semantic segmentation map of each part of the human body, wherein the joint points include body key points and body bone points; S64: Substituting the generated joint point map and semantic segmentation map of the target human body, the body skeleton points and key point information into the second neural network after deep learning to regress the human body posture and body shape parameters; S65: Obtain output 3D human body parameters, the parameters including 3D human body action POSE parameters and 3D human body shape SHAPE parameters; S7: inputting the obtained sets of posture and body shape three-dimensional human body parameters into a three-dimensional standard human body model for fitting; In which, during the movement of the 3D human body model, cloth simulation calculation is performed on the clothes worn by the human body model. The cloth simulation adopts a combination of cloth simulation and skinning methods. The skinning method is used for the part of the clothes that basically does not deform after the clothes move, and the cloth simulation method is used for the part that deforms during the movement of the clothes. During the cloth simulation process, when the human body model reaches the target posture, the gravity solution of the clothes is performed for several frames to ensure the realism of the clothes in the target posture. The moving process includes the following sub-steps: S81: Match the obtained 3D human body POSE and SHAPE parameters with several base and bone parameters of the 3D standard human body model; S82: inputting the obtained several groups of basis and bone parameters into a standard three-dimensional human body parameter model for fitting; S83: The three-dimensional human body model has a mathematical weight relationship between skeleton points and model meshes, and the determination of the skeleton points can be associated with the human body model of the target human body posture; S8: Obtain a target human body model having the same posture and body shape as the target human body and wearing the changed clothes.

2. The method according to claim 1, characterized in that It also includes two different skinning brush weighting steps between the 3D clothing model and the skeleton, and between the human model skin mesh and the skeleton, to ensure an improved motion matching relationship between the clothing, the human skin mesh and the skeleton.

3. The method according to claim 1, characterized in that When the model is wearing the clothing, the normal fitting state of the three-dimensional clothing model relative to the standard human body model grid is expanded to a certain size, so that the penetration phenomenon of the three-dimensional clothing model is reduced.

4. The method according to claim 1, characterized in that: The fitting step of step S4 further includes the following sub-steps: S41: Obtaining the position coordinates of the initial posture and the target posture; S42: Generate an animation sequence moving from an initial posture to a target posture; S43: In the process of generating the animation sequence, a grid interpolation method is used for processing; S44: The interpolation speed is set so that the distance between the initial point and the target point is slow, and the intermediate movement process is fast; S45: when driving to the final target posture, the animation sequence is obtained by stopping for several frames; S46: Complete driving the skeleton to move from the initial posture to the target posture.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 4 are implemented.

6. An electronic device, characterized in that: The invention comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the method steps described in any one of claims 1 to 4 when executing the program stored in the memory.

Images