A three-dimensional human body model, a reconstruction method thereof and a human body data measuring method
By training the SMPL model with human images acquired from multiple angles and optimizing shape and pose parameters, the problem of relying on expensive equipment and non-parametric models in existing technologies is solved, and low-cost, adjustable 3D human reconstruction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PHOTONICS INTEGRATION (WENZHOU) INNOVATION RES INST
- Filing Date
- 2023-08-25
- Publication Date
- 2026-06-19
AI Technical Summary
Existing 3D human body reconstruction technology relies on expensive hardware and the reconstructed model is non-parametric, making it impossible to change the model's shape and posture by adjusting a few parameters.
By acquiring multi-angle human images, extracting key point information, training the SMPL model, optimizing shape and pose parameters, and reconstructing a parametric 3D human model.
It has achieved a low-cost, widely applicable parametric 3D human body model that can adjust posture and shape as needed.
Smart Images
Figure CN117115350B_ABST
Abstract
Description
Technical Field
[0001] This invention discloses a three-dimensional human body model and its reconstruction method, as well as a human body data measurement method, belonging to the field of human body recognition technology. Background Technology
[0002] 3D human body reconstruction obtains a 3D model and data of the human body through optical measurement, and is widely used in fields such as game animation production, motion analysis, virtual try-on, medical health, and interactive entertainment. Currently, 3D human body reconstruction technology is based on a 360-degree rotating robotic arm and a depth camera mounted on the arm. This allows for the capture of depth images of the human body from different perspectives. Point cloud data of the human body is obtained from these depth images. After denoising and registration of the point cloud data, an initial 3D human body model is constructed using the point cloud data and a 3D human body measurement system. Finally, surface reconstruction is performed on the surface of the initial 3D human body model to obtain a high-precision 3D human body model.
[0003] However, when implementing the above-mentioned technologies, the quality of the depth images affects the accuracy of the 3D human body model, making the existing technologies heavily reliant on relatively expensive depth cameras and bulky hardware robotic arms, which renders them unusable in many situations. At the same time, the 3D human body model reconstructed in the existing technologies is a non-parametric model, and therefore does not have the ability to change the shape and posture of the model by adjusting a few parameters. Summary of the Invention
[0004] The purpose of this application is to provide a three-dimensional human body model, a method for its reconstruction, and a method for measuring human body data to solve the technical problem that existing three-dimensional human body models are non-parametric and therefore lack the ability to change the shape and posture of the model by adjusting a few parameters. To achieve the above objective, this invention proposes a three-dimensional human body model, a method for its reconstruction, and a method for measuring human body data, the specific solutions of which are as follows:
[0005] A method for reconstructing a three-dimensional human body model includes the following steps:
[0006] Step 1: Obtain M orientation images of the human body from different angles and extract key point information of the human body from each orientation image to obtain M sets of key point information of the human body, where M≥4;
[0007] Step 2: Train the SMPL model according to the human key point information in the M groups to obtain M trained SMPL models;
[0008] Step 3: Reconstruct a 3D human body model based on M trained SMPL models.
[0009] Preferably, SMPL models are trained based on the M groups of human keypoint information to obtain M trained SMPL models, specifically including:
[0010] Step 2.1: Input the key point information of each group of human bodies as the true human body values into the SMPL model;
[0011] Step 2.2: Determine each trained SMPL model based on the output of each trained SMPL model, and obtain M trained SMPL models.
[0012] Preferably, each trained SMPL model is determined based on the output of the SMPL model during each training session, specifically including:
[0013] Extract the second human key point information mapped by the SMPL model in each training session, and obtain the difference between the second human key point information and its corresponding real human value;
[0014] When the absolute value of the difference is less than the first threshold, the SMPL model in training is denoted as the trained SMPL model.
[0015] Preferably, the key human body information includes the human body outline and human body joints.
[0016] Preferably, a 3D human body model is reconstructed based on M trained SMPL models, specifically including:
[0017] Step 3.1: Extract the shape parameters of each trained SMPL model to obtain M sets of shape parameters and construct the parameter space;
[0018] Step 3.2: Extract multiple shape parameters that meet preset conditions within the parameter space, and reconstruct a three-dimensional human body model based on the multiple shape parameters.
[0019] Preferably, the preset condition is:
[0020] The multiple shape parameters minimize the sum of the absolute values of the differences between the contour maps projected by the M trained SMPL models and their corresponding real human contour maps.
[0021] A three-dimensional human body model, a three-dimensional human body model reconstructed based on any of the aforementioned three-dimensional human body model reconstruction methods.
[0022] A method for measuring human body data from a three-dimensional human body model, and adjusting the human posture based on the three-dimensional human body model;
[0023] Key points are located on the three-dimensional human body model after the posture is adjusted, and the three-dimensional human body cross section is obtained according to the circumferential cutting method.
[0024] The actual human body data is determined based on the three-dimensional human body cross-section.
[0025] Preferably, determining real human body data based on the three-dimensional human body cross-section specifically includes:
[0026] Human data of the three-dimensional human body model is obtained based on the three-dimensional human body cross-section;
[0027] The human body data is converted into real human body data.
[0028] Beneficial effects: This invention does not require bulky and expensive special equipment, its cost is low, and it can be widely used in a variety of occasions. At the same time, the three-dimensional human body model reconstructed by this invention is a parametric model, and the posture and shape can be adjusted parametrically as needed. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the three-dimensional human body model reconstruction and human body data measurement process according to an embodiment of the present invention;
[0030] Figure 2 The following are schematic diagrams of the loss of the three-dimensional human body model in various directions in the embodiments of the present invention: (a) is a schematic diagram of the loss of the SMPL model and the real human body key points in direction a in the embodiments of the present invention, and (b) is a schematic diagram of the loss of the SMPL model and the real human body key points in direction b in the embodiments of the present invention.
[0031] Figure 3 This is a schematic diagram of the pose of the three-dimensional human body model A in an embodiment of the present invention;
[0032] Figure 4 This is a schematic diagram of the changes in the three-dimensional human body cross-section in an embodiment of the present invention, wherein (c) is the three-dimensional human body cross-section above the armpit point in an embodiment of the present invention, and (d) is the three-dimensional human body cross-section below the armpit point in an embodiment of the present invention. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of the invention.
[0034] Example 1
[0035] like Figure 1 As shown, a method for reconstructing a three-dimensional human body model includes the following steps:
[0036] Step 1: Obtain M orientation images of the human body from different angles and extract key point information of the human body from each orientation image to obtain M sets of key point information of the human body, M≥4; the key point information of the human body includes the human body outline and human body joints.
[0037] Specifically, a mobile phone is used to capture 360-degree horizontal video images of the test subject's body at a preset distance. This video data is then used to extract 36 azimuth images of the body from different angles. Each azimuth image describes the morphological features of the body from different perspectives. The background of each azimuth image is then removed, and key human point information, including the body outline and joints, is extracted. The preset distance between the body and the mobile phone is determined based on the desired imaging effect. The body is positioned in the center of the video image; the body occupies at least two-thirds of the image area; the rotation speed of the body in the video image ranges from 15 to 20 seconds per revolution; and the video image resolution is at least 720P. To ensure accuracy, the test subject needs to wear tight-fitting clothing and the sampling is conducted under good lighting conditions.
[0038] Step 2: Train the SMPL model according to the human key point information in the M groups to obtain M trained SMPL models;
[0039] Step 2.1: Input the key point information of each group of human bodies as the true human body values into the SMPL model;
[0040] Step 2.2: Extract the second human keypoint information mapped by the SMPL model in each training session, and obtain the difference between the second human keypoint information and the corresponding real human value; when the absolute value of the difference is less than a first threshold, the SMPL model is determined as the trained SMPL model, and M trained SMPL models are obtained.
[0041] Specifically, it includes the following steps:
[0042] S1: Extract 36 sets of human body key point information from 36 orientation maps. The human body key point information is the human body outline and human body joints. In this embodiment, it includes the human body outline mask and human body joints.
[0043] S2: To reduce the training process of the model, basic information such as the height, gender, and weight of the testers is obtained, and the above basic information is used to generate a standard SMPL model that is closer to the testers.
[0044] S3: Input the human body key point information obtained in S1 as the true human body value into the standard SMPL model;
[0045] S4: Extract the second human keypoint information mapped by the standard SMPL model;
[0046] S5: Compare the difference between the second human body keypoint information in S3 and S4 and the actual human body value, and use the difference to construct a loss function to train the model. The loss function is as follows:
[0047] E(β, θ) = E J (β,θ)+E silh (β, θ)
[0048] In the formula, E J (β, θ) represents the human joint loss function, E silh (β, θ) represents the human contour loss function, where β represents the shape parameter of the SMPL model and θ represents the pose parameter of the SMPL model.
[0049] S6: Adjust the parameters of the standard SMPL model, including shape parameters and pose parameters. Repeat S3 to S5 until the absolute value of the loss function E(β, θ) is less than a set first threshold, the first threshold being in the range of 0.001 to 0.01.
[0050] Specifically, in this embodiment, the parameters are adjusted until the absolute value of the loss function approaches 0, specifically less than 0.01. The closer E(β, θ) is to 0, the closer the trained SMPL model is to the real human body shape in the current orientation.
[0051] Specifically, such as Figure 2 As shown, in this embodiment, the loss includes the SMPL model and the real human keypoint loss under the orientation corresponding to two orientation maps, a and b. White dots represent human keypoints in the SMPL model, and black dots represent the real human keypoints of the test subject. First, based on the constructed loss function E... J (β, θ) Obtain the calculation results of the human joint loss function, adjust the shape parameters and pose parameters of the SMPL model, and continuously iterate and optimize to make E J As (β, θ) approaches 0, the black and white points increasingly overlap. Secondly, based on the constructed loss function E... silh (β, θ) yields the calculation results of the human contour loss function, making the contour in the SMPL model continuously approximate the contour of the real human body. The area of the contour in the model that is smaller than the contour of the real human body represents a human contour loss. Figure 2 As can be seen, the area of blank space around the human body represents another form of loss of human contour. After multiple parameter adjustments and optimizations, E... silh The smaller the absolute value of (β, θ), the less white space is left at the edge of the human body, and the higher the degree of overlap between the contour in the model and the contour of the real human body.
[0052] S7: Finally, 36 SMPL models that closely resemble the real human body in the corresponding orientation map are generated, i.e., 36 trained SMPL models. The 36 trained SMPL models correspond to 36 sets of shape parameters and pose parameters, thus completing the pose reconstruction of the SMPL model in each orientation.
[0053] Step 3: Reconstruct a 3D human body model based on M trained SMPL models.
[0054] Step 3.1: Extract the shape parameters of each trained SMPL model to obtain M sets of shape parameters and construct the parameter space;
[0055] Step 3.2: Extract multiple shape parameters that meet preset conditions within the parameter space, and reconstruct a three-dimensional human body model based on the multiple shape parameters.
[0056] The multiple shape parameters minimize the sum of the absolute values of the differences between the contour maps projected by the M trained SMPL models and their corresponding real human contour maps.
[0057] Specifically, the 36 trained SMPL models are combined, that is, the SMPL model combination optimization is carried out in multiple directions. The specific combination process includes extracting the shape parameters of each trained SMPL model. In this embodiment, the shape parameters of each trained SMPL model consist of 10 scalars, that is, extracting the shape parameters of 36 trained SMPL models. Each trained SMPL model includes 10 shape parameters, that is, extracting 36*10 shape parameters.
[0058] The parameter space is constructed from 36*10 parameters. Ten shape parameters of arbitrary combination are extracted from the parameter space to form a set of shape parameters β.
[0059] The optimal set of shape parameters β is found by minimizing the J value, and a 3D human body model is reconstructed using this optimal set of shape parameters. The J value is calculated using the following formula:
[0060]
[0061] In the formula, M silh (β,θ i () represents a 3D human body model composed of a set of shape parameters β and the i-th pose parameter, which projects a contour map in the i-th pose. silh (i) represents the actual human silhouette of the tester in the i-th pose.
[0062] By finding a set of shape parameter combinations β in the parameter space that minimizes the squared difference between the contour map projected by the model in these 36 poses and the contour map of the real person, the optimal set of shape parameters β is obtained. The optimal set of shape parameters β is used to reconstruct the three-dimensional human body model.
[0063] Example 2
[0064] A three-dimensional human body model, a three-dimensional human body model reconstructed based on the three-dimensional human body model reconstruction method of Embodiment 1.
[0065] Example 3
[0066] A method for measuring human body data from a 3D human body model includes: adjusting the human posture in the 3D human body model as described in Embodiment 2; locating key points on the 3D human body model after posture adjustment; obtaining a 3D human body cross-section using the circumferential cutting method; determining real human body data based on the 3D human body cross-section; obtaining human body data from the 3D human body model based on the 3D human body cross-section; and converting the human body data into real human body data.
[0067] Specifically, such as Figure 3 As shown, in order to facilitate the search for human joint points, such as armpits, buttocks, shoulders, and waists, the 3D human model is converted into an A pose, that is, the model pose is adjusted to an A pose.
[0068] First, based on the human body proportions, the approximate locations of various joint points on the human body are determined using a fuzzy positioning method, as shown in Table 1:
[0069] Table 1
[0070]
[0071]
[0072] Secondly, the joints in the three-dimensional human body model are accurately located using the local circumferential cutting method;
[0073] Taking armpit positioning as an example, in posture A, the approximate area of the armpit point is obtained according to Table 1. At 0.75H from the armpit point, the approximate area is obtained. Specifically, in this embodiment, a series of three-dimensional human body cross-sections are obtained by horizontally cutting across the three-dimensional human body from top to bottom within the range of 0.76H to 0.74H. For example... Figure 4 As shown, a single region in figure (c) becomes (d). Figure 3 The critical point of each region is the precise axillary point.
[0074] Taking crotch height as an example, the human body is horizontally cut from top to bottom in the approximate area of the buttocks point to obtain a series of three-dimensional human body cross sections, and the crotch height is located by the changing trend.
[0075] Finally, other data information is converted based on the actual height and the model height.
[0076] The reconstructed 3D human body model is transmitted to the mobile APP terminal display module in standard obj and ply formats, and the converted 3D human body data and other information are displayed.
[0077] This invention creates a 3D human body model using ordinary visual images and measures human body data from the 3D human body model, achieving a simple and low-cost data acquisition device. Users only need a mobile phone to operate this invention, making it easy to use and eliminating the need for bulky equipment. The 3D model achieved by this invention is based on parametric representation and can change its posture and shape according to specific needs, which can be further applied to multiple fields such as personalized clothing customization platforms, game animation production, motion analysis, simulated try-on, medical health, and interactive entertainment.
[0078] The above embodiments illustrate only one implementation of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.
Claims
1. A method for reconstructing a three-dimensional human body model, characterized in that, Includes the following steps: Step 1: Obtain M orientation images of the human body from different angles and extract key point information of the human body from each orientation image to obtain M sets of key point information of the human body, where M≥4; Step 2: Train SMPL models based on the human keypoint information in the M groups to obtain M trained SMPL models, specifically including: Step 2.1: Input the key point information of each group of human bodies as the true human body values into the SMPL model; Step 2.2: Determine each trained SMPL model based on the output of each trained SMPL model, and obtain M trained SMPL models; Step 3: Reconstruct a 3D human body model based on M trained SMPL models, specifically including: Step 3.1: Extract the shape parameters of each trained SMPL model to obtain M sets of shape parameters and construct the parameter space; Step 3.2: Extract multiple shape parameters that meet preset conditions within the parameter space, and reconstruct a 3D human body model based on the multiple shape parameters; the preset condition is: the multiple shape parameters minimize the sum of the absolute values of the differences between the contour maps projected by the M trained SMPL models and their corresponding real human body contour maps, specifically: M=36, combining 36 trained SMPL models, i.e., multi-faceted SMPL model combination optimization. The specific combination process includes extracting the shape parameters of each trained SMPL model. Each trained SMPL model's shape parameters consist of 10 scalars; that is, extracting the shape parameters of all 36 trained SMPL models. Each trained SMPL model includes 10 shape parameters. One shape parameter; Will The shape parameters constitute a parameter space. Any combination of 10 shape parameters can be extracted from this parameter space to form a set of shape parameters. ; By minimizing The optimal set of shape parameters is found. Using the optimal set of shape parameters Reconstructing a 3D human body model; The value is calculated using the following formula: ; In the formula, Represented by a set of shape parameters With the A three-dimensional human body model composed of several pose parameters, which projects the first... Contour images in various poses Indicates that the testers were at the A realistic human silhouette shown in various poses.
2. The method for reconstructing a three-dimensional human body model according to claim 1, characterized in that, Each trained SMPL model is determined based on the output of the SMPL model during each training session, specifically including: Extract the second human key point information mapped by the SMPL model in each training session, and obtain the difference between the second human key point information and its corresponding real human value; When the absolute value of the difference is less than the first threshold, the SMPL model in training is denoted as the trained SMPL model.
3. The method for reconstructing a three-dimensional human body model according to claim 1, characterized in that, The key information about the human body includes the human body outline and human body joints.
4. A three-dimensional human body model, characterized in that, A three-dimensional human body model reconstructed based on the reconstruction method of the three-dimensional human body model according to any one of claims 1-3.
5. A method for measuring human body data from a three-dimensional human body model, characterized in that, Adjusting the human posture in the three-dimensional human body model based on claim 4; Key points are located on the three-dimensional human body model after the posture is adjusted, and the three-dimensional human body cross section is obtained according to the circumferential cutting method. The actual human body data is determined based on the three-dimensional human body cross-section.
6. The method for measuring human body data of a three-dimensional human body model according to claim 5, characterized in that, Determining real human body data based on the aforementioned three-dimensional human body cross-section specifically includes: Human data of the three-dimensional human body model is obtained based on the three-dimensional human body cross-section; The human body data is converted into real human body data.