Three-dimensional human body point cloud reconstruction and automatic measurement method based on multi-view image

Abstract

The invention discloses a three-dimensional human body point cloud reconstruction and automatic measurement method based on a multi-view-angle image, and the method specifically comprises the steps: 1, obtaining a virtual multi-view-angle human body image and a real multi-view-angle human body image collected in a real scene, and enabling the two types of data to be collectively called multi-view-angle human body images; 2, generating a corrected multi-view human body image, a sparse point cloud and camera parameters, converting formats, performing densification processing, and suppressing background interference in combination with a semantic segmentation mask to generate a dense human body point cloud P; step 3, outputting standardized posture human body grids and corresponding parameters; and step 4, completing automatic human body measurement and outputting a structured file and a visual result. The method solves the problems that in the prior art, a multi-view reconstruction mechanism oriented to human body structure features is lacked, the coverage of an acquisition mode is insufficient, and the parameterized fitting precision is not high.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and 3D reconstruction technology, specifically involving a method for 3D human point cloud reconstruction and automatic measurement based on multi-view images. Background Technology

[0002] With the rapid development of computer vision and deep learning technologies, human body size measurement based on 3D point clouds has gradually become an important research direction in fields such as intelligent manufacturing, healthcare, virtual fitting, and security identification. The accuracy of 3D human body measurement directly affects its feasibility in practical applications; therefore, high-precision, low-cost human body reconstruction and automatic measurement technologies have gradually become research hotspots. Existing 3D human body reconstruction methods mainly fall into two categories: those based on high-precision scanners and those based on image-based deep learning methods. The former (such as structured light or laser scanning systems) has high accuracy and robustness, but the equipment is expensive, bulky, and has strict requirements for the acquisition environment, making it difficult to apply to everyday scenarios or mobile devices. Moreover, the acquisition process is complex and costly. The latter has developed rapidly in recent years, with typical methods such as MVSNet, R-MVSNet, and PatchMatchNet achieving high-precision 3D reconstruction of general objects. However, the human body surface has strong non-rigid characteristics and is significantly affected by posture, clothing occlusion, and lighting, making it difficult for existing MVS methods to directly generate accurate dense human body point clouds. Some studies have attempted to generate human body meshes from single images using parametric models (such as SMPL and SMPL-X) or implicit function networks (such as PIFuHD, PaMIR, and ICON). However, the outputs are mostly implicit surfaces or parametric meshes, failing to provide dense point cloud data directly for geometric measurements. These limitations make it difficult for image-based low-cost 3D anthropometrics to simultaneously achieve detail accuracy and shape realism. Furthermore, direct point cloud-based anthropometrics still faces several challenges. Differences in human pose, clothing occlusion, and environmental noise often lead to missing or distorted point clouds, affecting the accuracy of key dimension extraction. Because point clouds lack explicit topological structure, it is difficult to directly locate anatomical landmarks. To address these challenges, researchers have proposed fitting point clouds to parametric human models to achieve standardized measurements of key dimensions and girths. However, existing methods largely rely on scanner point cloud data and lack a complete, low-cost human reconstruction and measurement solution for general multi-view image input.

[0003] In summary, current research in the field of human body 3D point cloud reconstruction and measurement still has three shortcomings: First, high-precision human body point clouds currently rely heavily on 3D scanning equipment. Although the accuracy is high, the equipment is expensive and complex to operate, making it unsuitable for ordinary scenarios and large-scale applications. Furthermore, existing multi-view reconstruction models are mostly designed for general objects and lack point cloud reconstruction mechanisms tailored to human structural features, making it difficult to obtain high-quality human body point clouds at low cost. Second, the acquisition methods are mostly single-circle surround shooting, leading to missing details and proportional deviations in the reconstructed model at local body points. Third, parametric model fitting does not fully incorporate point cloud constraints, resulting in discrepancies between the generated human body mesh and the actual shape, affecting the accuracy and consistency of automatic measurement. Summary of the Invention

[0004] The purpose of this invention is to provide a method for three-dimensional human point cloud reconstruction and automatic measurement based on multi-view images, which solves the problems of lack of multi-view reconstruction mechanism oriented to human structural features, insufficient coverage of acquisition methods, and low accuracy of parameter fitting in the prior art.

[0005] The technical solution adopted in this invention is: The method for 3D human point cloud reconstruction and automatic measurement based on multi-view images is as follows: Step 1: Acquire virtual multi-view human images and real multi-view human images collected in real scenes, both of which are collectively referred to as multi-view human images; Step 2: Generate calibrated multi-view human images, sparse point clouds, and camera parameters, convert their formats, perform densification processing, and combine semantic segmentation masks to suppress background interference, generating a dense human point cloud P; Step 3: Generate an initial SMPL-X human mesh M. 0 Based on the parameters, and combined with the dense human point cloud P generated in step 2, a non-standard human pose mesh M* is obtained through optimization. The non-standard human pose mesh M* is then subjected to pose normalization processing to output a normalized pose human mesh. and corresponding parameters; Step 4: Complete the automated anthropometric measurements and output structured documents and visualization results.

[0006] The invention is further characterized in that: in step 1, the publicly available THuman2.0 dataset is first selected as the data source for virtual multi-view human images. The complete human .obj model and its materials and textures in the dataset are loaded into the Taichi rendering system to generate multi-view human images corresponding to the same human body. Then, the human .obj model is divided into different regions of the body parts, and virtual camera trajectories are planned for each region, so that the Taichi rendering system can render segmented images obtained from different regions of the body parts. In addition to virtual rendering, real multi-view human images are also collected in real scenes. The subject stands on an automatic rotating platform, which completes a 360° rotation at a uniform angular velocity. A liftable camera device records video sequences at different elevation angles according to different regions of the body parts. After collection, the video is parsed into static image frames using ffmpeg to construct a real multi-view image set. Finally, the virtual multi-view human images generated based on the THuman2.0 dataset and the real multi-view human images collected in real scenes are obtained. Both types of data are collectively referred to as multi-view human images.

[0007] Step 2 is implemented as follows: Step 2.1, using the incremental structure of the COLMAP sparse reconstruction algorithm, the corrected multi-view human images, camera parameter files, and sparse point cloud files are generated from the motion method; Step 2.2, combining the corrected multi-view human images, sparse point cloud files, and camera parameter files output in Step 2.1, the MDF-Net deep learning 3D reconstruction network model is used to achieve multi-view stereo vision dense reconstruction.

[0008] Step 2.1 specifically involves: Step 2.1.1: Perform feature extraction and feature matching on the multi-view human images processed in Step 1 to obtain matching feature point pairs for each multi-view human image pair; then select initial image pairs from the multi-view human image pairs, selecting image pairs with a single image pair matching feature point pair number ≥ 1500 and a valid feature point pair number ≥ 800 after removing mismatches, and with the camera baseline in the range of 0.3m~1.5m. Finally, determine the multi-view human image pairs that meet the requirements as initial image pairs; without given camera intrinsic parameters, estimate the camera intrinsic parameter matrix using the simplified Zhang calibration method based on the matching feature point pairs of the initial image pairs, and obtain the focal length, principal point coordinates, and lens distortion parameters. The camera intrinsic parameter matrix is ​​expressed as follows: ,in Focal length , The principal point position is determined in pixel coordinates; subsequently, a lens distortion model is constructed based on the camera intrinsic parameter matrix, and geometric correction is performed on the multi-view human images processed in step 1 to eliminate image distortion, outputting the geometrically corrected multi-view human images; the fundamental matrix is ​​obtained through feature matching. and using the formula Will Convert to ,in For the camera intrinsic parameter matrix, Transpose it. Based on the matrix, For the essential matrix; for the essential matrix Singular value decomposition yields four sets of candidate camera pose solutions, each of which fully includes the corresponding rotation matrix. Translation vector Subsequently, triangulation is performed on the matching feature point pairs of the initial image pair. The only feasible camera pose solution is selected based on the reprojection error; specifically, the pose solution with an average reprojection error ≤ 2.0 pixels and a reprojection error greater than 2.0 pixels accounting for no more than 5% of the feature point pairs is chosen as the only feasible solution. After solving, the first image in the initial image pair is set as the origin of the world coordinate system, the pose of the second image in the initial image pair is determined, and a projection matrix is ​​constructed. ,in Used to convert homogeneous three-dimensional coordinates Projecting onto the image coordinate system generates homogeneous pixel coordinates The projection relationship is expressed as The projection matrix constructed using the second image in the initial image pair. The homogeneous 3D coordinates X are projected onto the image coordinate system, and the initial sparse point cloud is triangulated to recover the initial sparse point cloud. Abnormal 3D points with reprojection errors greater than 2.0 pixels in the initial sparse point cloud are removed. Step 2.1.2: Add new images one by one based on the initial sparse point cloud. When the number of matching points between the new image and the current point cloud formed by iterative expansion of the initial sparse point cloud is no less than six, use the EPnP algorithm combined with RANSAC to eliminate outliers and estimate camera extrinsic parameters. For matching feature point pairs that are successfully matched with the new image and the current point cloud but have not yet generated 3D points, perform triangulation and eliminate outliers according to constraints, and expand the point cloud and camera pose set. After all multi-view human images are added, use global bundle adjustment combined with the sparse Levenberg-Marquardt algorithm and Schur complement decomposition to optimize the camera pose and 3D point coordinates, eliminate unqualified observation relationships and outlier 3D points, and finally output the corrected multi-view human images, camera parameter files, and sparse point cloud files using the COLMAP sparse reconstruction algorithm.

[0009] In step 2.2: The deep learning 3D reconstruction network MDF-Net model contains four functional modules: a 2D feature pyramid network module, a deep hypothesis and cost volume construction module, a probabilistic inference module, and a super-resolution network module. The specific structures of the four modules are as follows: The 2D feature pyramid network module adopts a shared weight convolutional backbone structure, divided into three feature extraction levels from top to bottom. The first level outputs a feature map with 64 channels and a resolution of 1 / 8 of the input image through 4 convolutions and downsampling with a stride of 2. The second level outputs a feature map with 32 channels and a resolution of 1 / 4 of the input image through 2 convolutions and downsampling with a stride of 2. The third level directly outputs a feature map with 16 channels and a resolution of 1 / 2 of the input image. The fusion structure adopts a top-down feature upsampling + lateral skip connection mode. The upsampling adopts bilinear interpolation, and the lateral connection is added element-wise to the corresponding level feature map after unifying the number of channels through 1×1 convolution. The depth hypothesis and cost volume construction module is a three-stage cascaded structure, consisting of a coarse depth estimation submodule, a Gaussian correction submodule, and a Laplacian refinement submodule. Each stage contains a three-level substructure of "feature projection - cost calculation - cost aggregation". Feature projection uses differentiable homography transformation to achieve feature distortion from the source view to the reference view. Cost calculation calculates the feature inner product between the reference view and the source view after performing softmax normalization on the grouped features. Cost aggregation concatenates the costs of all source views by channel, and finally outputs a 4-dimensional cost volume with the dimensions of "number of feature groups G × number of depth samples D × feature map height H × feature map width W". The probabilistic volume inference module is based on the 3D UNet symmetric architecture and is divided into an encoder and a decoder. The encoder contains four 3D convolutional blocks, each consisting of two 3×3×3 convolutional layers, a batch normalization layer, and a ReLU activation function layer. A 2×2×2 stride downsampling is performed after each convolutional block. The decoder is symmetrical to the encoder. Each decoder unit consists of one 2×2×2 transposed convolutional layer, a concatenation operation with the corresponding layer feature map of the encoder, two 3×3×3 convolutional layers, a batch normalization layer, and a ReLU activation function layer. The output is a probabilistic volume with the same size as the input cost volume. The super-resolution network module is based on the EDSR residual network architecture. The input layer is a 1×1 convolutional layer, and the intermediate layer consists of 6 consecutive residual blocks stacked together. Each residual block contains 2 layers of 3×3 convolutional layers + batch normalization layer + ReLU activation function layer. Shortcut skip connections are used inside the residual block. The output layer uses the PixelShuffle upsampling module, combined with a 3×3 convolutional layer to adjust the number of output channels. The final output is a feature map with the same resolution as the original input image.

[0010] Step 2.2 specifically involves: Step 2.2.1: Select one image from the corrected multi-view human images output in Step 2.1 as the reference view, and the rest as source views. Use a 2D feature pyramid network with shared weights to extract three sets of multi-scale features. The feature map channels are 64, 32, and 16, corresponding to resolutions of 1 / 8, 1 / 4, and 1 / 2 of the input image, respectively. These features are used in the four stages of coarse depth estimation, Gaussian correction, Laplacian refinement, and super-resolution. Based on the camera parameter file output in Step 2.1, in the four stages of coarse depth estimation, Gaussian correction, Laplacian refinement, and super-resolution, the source view features are projected onto the reference view using differentiable homography and bilinear interpolation based on the assumed depth. The projection points are calculated using the following formula: (1) In the formula, , , These are the camera intrinsic parameters, rotation matrix, and translation vector of the reference view, respectively. , , The first The corresponding parameters of each source view. Let be the homogeneous coordinates after projection. After completing the projection alignment of the source view features to the reference view, the aligned features of the reference view and all source views are uniformly divided into G groups according to the number of channels. First, the features of each group are normalized by softmax, and then the cost of the reference view and each source view is calculated for each group. The set of costs for all groups constitutes the first set. The cost body corresponding to each source view ; Step 2.2.2: The VCR-Net network then generates the weights for each source view. The VCR-Net network is a two-layer 3D CNN structure with 1×1 convolutional kernels. The specific structure is as follows: The input is a feature volume of dimension G×D×H×W, where G is the number of feature groups, D is the number of depth samples, and H and W are the height and width of the feature map, respectively. The first layer performs a 3D CNN convolution operation with a kernel size of 1×1×1 and a stride of 1. After convolution, it is processed by the ReLU activation function and batch normalized, outputting a feature volume of dimension 1×D×H×W. The second layer also performs a 3D CNN convolution operation with a kernel size of 1×1×1 and a stride of 1. After convolution, it is processed by the Sigmoid activation function without batch normalization, finally outputting a probability volume of dimension 1×D×H×W. The 1×D×H×W probability volumes of each source view are used as the weights for the second layer. Weight of each source view The cost volumes corresponding to all source views are weighted and aggregated to form the final cost volume. : (2) In the formula, Indicates the pixel position of the reference view. The ultimate cost, For the first The cost body corresponding to each source view. For the first The weights obtained after processing the source view costs using VCR-Net Total number of source views; Step 2.2.3: The three stages of coarse depth estimation, Gaussian correction, and Laplacian refinement all employ 3D UNet to regularize the cost volumes generated in each stage. The probability volumes for each stage are obtained using the softmax function, and the depth maps for each stage are obtained through soft argmin regression. Specifically, the coarse depth estimation stage combines the sparse point cloud file output from Step 2.1 to statistically analyze the depth distribution range of the sparse point cloud to verify and correct the preset maximum depth range [425.0 mm, 935.0 mm]. Within the corrected depth range, 48 depth planes are uniformly sampled to obtain the initial depth map. The Gaussian correction stage performs Gaussian distribution fitting based on the probability volume output from the coarse depth estimation stage, with the target curve being: (3) In the formula, As a deep hypothesis, This represents the probability volume value corresponding to the depth hypothesis. The mean parameter of the Gaussian distribution. The standard deviation parameter is the Gaussian distribution parameter; the new depth refinement interval is determined based on the fitted target curve. ,in To estimate the stage regression depth value for the coarse depth map, For the fitting parameters, The probability threshold is used, and 24 depth planes are uniformly sampled within the depth refinement interval; the Laplace refinement stage fits the Laplace distribution based on the probability volume output from the Gaussian correction stage, and the target curve for fitting is: (4) In the formula, It is a depth hypothesis The corresponding probability value, The depth value obtained by Gaussian-corrected regression. This is the scale parameter of the Laplacian distribution. The parameter is also solved using logarithmic transformation and least squares method, and a narrower depth interval is used to obtain a high-precision depth map; the refinement interval can be as small as 1mm-3mm. In the super-resolution stage, the cost volume is no longer constructed; instead, a lightweight super-resolution network based on the EDSR structure is used to upsample the high-precision depth map output from the Laplacian refinement stage to the original resolution. A high-resolution depth map is then reconstructed through convolution, residual blocks, and PixelShuffle. Step 2.2.4: Introduce a human semantic segmentation module into the deep learning 3D reconstruction network MDF-Net model to generate a high-precision human binary segmentation mask. Specifically: First, the human semantic segmentation module performs channel normalization on the corrected multi-view human images: grayscale images are converted to BGR format, PNG images are converted to BGR format after removing the alpha channel, and JPG / JPEG images are converted to RGB format. Then, the normalized images are converted to PIL format and input into the rembg network to extract the human foreground, outputting an RGBA image containing the alpha channel, where white represents the foreground and black represents the background. By extracting the alpha channel of this RGBA image and converting it to a PyTorch tensor, the pixel values ​​are binarized to generate a human binary segmentation mask. The definition is as follows: (5) In the formula, Simultaneously for pixels Human body binary segmentation mask value, The pixel position is used as a reference in the view; this is combined with a photometric consistency mask. With geometric consistency mask Perform pixel Joint screening, photometric consistency mask The definition is as follows: (6) In the formula, For the pixel in the i-th view The intensity value, Let be the intensity value of the corresponding pixel in the j-th view. For projection function, For pixels The corresponding three-dimensional point coordinates Preset photometric consistency threshold; geometric consistency mask The definition is as follows: (7) In the formula, For the pixel in the i-th view The intensity value, Let be the depth value of the corresponding pixel in the j-th view. For projection function, For pixels The corresponding three-dimensional point coordinates A preset geometric consistency threshold is set; photometric consistency mask is used. Geometric consistency mask Human body binary segmentation mask By applying joint constraints, the optimized pixel validity mask is obtained, as shown in the following formula: (8) In the formula, For pixels The final validity mask value, " is an element-wise multiplication operation; finally, it is fused to generate a dense human point cloud P, and the 3D reconstruction result is output in .ply format.

[0011] Step 3 specifically involves: Step 3.1: First, use the PyMAF tool to perform initial SMPL-X fitting on the corrected multi-view human images output in Step 2.1. The PyMAF tool can quickly regress the shape parameters of the SMPL-X model based on a single corrected multi-view human image's frontal view. Attitude parameters Translation parameters and the SMPL-X initial human body mesh M without clothing detail standards 0 The fitting process then minimizes the initial human body mesh M of SMPL-X. 0 The L1 loss parameters for iteratively optimizing the rendered human body contour and image contour are as follows: (9) In the formula, For L1 loss, Initial human body mesh M for SMPL-X 0 The rendered human body outline For the image contour, iterate 3-5 times until the loss converges, and finally output the SMPL-X initial human mesh M that matches the pose and shape of the input human image. 0 ; Step 3.2, using the SMPL-X initial human body mesh M generated in step 3.1 0 Based on this, and combined with the dense human point cloud P generated in step 2, the initial human mesh M of SMPL-X is jointly optimized through multi-stage layering and multi-constraint optimization. 0 The details are as follows: First, create the initial human body mesh M using SMPL-X. 0A rigid transformation is used to align the center of the mesh with the center of the dense human point cloud P, achieving complete global alignment. Then, an iterative nearest-point algorithm is introduced for fine-grained coarse alignment, using Chamfer Distance as the alignment metric. The rotation and translation parameters of the mesh are iteratively optimized to ensure the initial human mesh M... 0 The global pose deviation from the dense human point cloud P is less than 5°; based on the shape parameters of the SMPL-X model Attitude parameters Translation parameters To optimize the variables, a multi-constraint optimization objective is constructed, including geometric alignment, topological consistency, and human prior. The specific constraints are as follows: Chamfer Distance is used to measure the initial human grid M. 0 The difference between vertices and the dense point cloud P is expressed as: (10) In the formula, For Chamfer Distance loss, For grid vertices, For points in a point cloud, It is the number of grid vertices. This refers to the number of points in the point cloud. Then, a normal consistency constraint is introduced. This is used to constrain the directional consistency between the vertex normals of the human body mesh and the normals of the dense point cloud. The normal consistency loss is: (11) In the formula, Let be the normal vector of the grid vertex. Let be the normal vector of the point cloud. The pose prior constraint uses the joint rotation angle as a variable to impose kinematic restrictions on each joint; its loss form is defined as: (12) In the formula, For attitude prior loss, Let K be the rotational parameter of the k-th joint, where K is the number of joints. Here is the joint angle penalty function. Prior shape constraints are applied to ensure the shape parameters reside in a reasonable shape space; their loss is defined as: (13) In the formula, For the prior loss of form, This is the shape parameter vector of SMPL-X. Its norm is 2; the local Laplacian smoothing term is used to normalize the surface variation of the mesh in the local neighborhood, and its loss is defined as: (14) In the formula, For local Laplacian smoothing loss, For the i-th grid vertex, Let j be the set of vertices in the neighborhood of vertex i, and j be the set of vertices in the neighborhood. any vertex index in the array, Let be the coordinates of the j-th vertex in the neighborhood; the mesh self-intersection penalty term is used to detect and eliminate the interpenetration relationship between triangular faces that occurs during the optimization process, and its loss is defined as: (15) In the formula, For grid self-intersection penalty loss, and For any two distinct triangular facets, This indicates a pair of facets that intersect or cross each other. This is the self-intersection penalty function. Combining all constraints, we obtain the overall objective function. : (16) In the formula, It is the weight of Chamfer Distance. It is the weight of the normal consistency loss. These are the weights of the attitude prior constraints. These are the weights of the prior constraints on the shape. These are the weights of the local Laplace smoothing constraint. These are the weights of the mesh self-crossing penalty loss. The mesh parameters are iteratively updated using the Adam optimizer. The update rules are as follows: (17) In the formula, For learning rate, It is the gradient of the loss function with respect to the grid parameters; After iteration, a non-standard human pose mesh M* that fits the shape of the dense human point cloud P is obtained; Step 3.3: Convert the fitted non-standard human pose mesh M* into a standard T-pose, as follows: Extracting pose parameters from a non-standard human pose mesh M* Separate the global pose and local joint pose, and construct the standard pose parameters for the T-pose. The root joint rotation is set to an identity matrix. Both arm joints are rotated to a horizontally extended position, and both leg joints are rotated to a straight and joined position. The rotations of the remaining joints are set to a neutral position. Then, based on the SMPL-X skin weights, a linear hybrid skinning algorithm is used to adjust the standard pose parameters. The formula applied to the fitted shape mesh is as follows: (18) In the formula, Let be the skinning weight of joint j corresponding to vertex i. for The rotation matrix of joint j. Let J be the position of joint j in M*. To determine the standard position of joint j in pose T, the transformed pose mesh is locally smoothed to remove vertex overlap or surface wrinkles that may occur during pose transformation, resulting in the final standardized pose human mesh. and its corresponding model parameters.

[0012] Step 4 is as follows: First, based on the key point definition automatic measurement algorithm provided by the SMPL-X official documentation, the standardized pose human mesh generated in step 3 is loaded. The model parameters and the difference between the maximum and minimum values ​​of the grid vertices on the Y-axis of the world coordinate system are used to obtain the height. Then, key dimensions are measured: based on the model parameters, the 3D coordinates of 24 joint points are calculated, and then the linear dimensions of shoulder width, left and right arm length, and left and right leg length are measured. For circumference measurement, the cross-sectional search range is first located by joint points, and the hip circumference cross-section with the largest area is found by sliding the virtual horizontal plane. Combined with the spinal joint points, the chest and waist circumference cross-sections are determined. The vertices of each cross-section are extracted and projected onto the XZ plane. The convex hull algorithm is used to calculate the perimeter of the cross-section, completing the chest, waist, and hip circumference measurements. Simultaneously, ten key landmark points are dynamically generated according to the definition of clothing measurement, including six feature points on the left and right shoulders, left and right hips, and chest and waist cross-sections. Through initial joint point positioning, finding grid intersections by ray projection along the Z-axis, cross-sectional geometric screening, symmetry mirroring, and linear interpolation, the landmark points are ensured to be located on the human body surface and satisfy symmetry. Finally, the key dimension indicators and the 3D coordinates of the landmark points are saved as JSON or TXT structured files, generating two visualization charts: one showing the human body grid, measurement cross-sections, and joint points, and the other highlighting the ten key landmark points.

[0013] The beneficial effects of this invention are: This invention proposes an end-to-end method for 3D human point cloud reconstruction and automatic measurement based on multi-view human images. First, the method proposes a segmented multi-view data acquisition strategy, dividing the human body into multiple local regions for separate imaging, which effectively improves the accuracy of local details and ensures the integrity and precision of the reconstructed point cloud. Second, the introduction of a human semantic segmentation mask in the dense reconstruction stage can suppress background noise and false contours, improving the accuracy and structural coherence of the human region point cloud, thereby improving photometric consistency and geometric constraint effects, providing stable input for subsequent measurements. Third, the method utilizes PyMAF to generate initial... The human body mesh adaptively predicts and matches the mesh structure to the human body shape in the image, making the initial human body mesh more closely resemble the real human form. Then, by introducing point cloud constraints to optimize the initial human body mesh, the matching accuracy between the generated mesh and the real human body shape can be further improved, ensuring the accuracy of the measurement results. Finally, the optimized mesh is subjected to T-pose normalization processing, and key measurement parameters such as height, shoulder width, and chest circumference, as well as ten clothing fit markers on the upper body, are automatically extracted. This achieves end-to-end, low-cost, and high-precision 3D human body point cloud reconstruction and automatic measurement, providing reliable technical support for customized clothing, personalized design, and the construction of human body databases. The method of this invention solves the problems of existing technologies, such as the lack of a multi-view point cloud reconstruction mechanism oriented towards human body structural features, insufficient coverage of acquisition methods, and low accuracy of parameterized fitting. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention; Figure 2 A diagram illustrating data preparation and segmented rendering; Figure 3 This is an illustration of a scene for capturing selfies. Figure 4 To reconstruct the network architecture diagram for dense network; Figure 5 This is a schematic diagram of the maskless reconstruction result; Figure 6 This is a schematic diagram of the reconstruction result with a mask. Figure 7 Comparison of reconstruction results from different perspectives and quantities in a single segment; Figure 8 The ground truth graph for the person coded as 0021 in the THuman2.0 dataset; Figure 9 This is the reconstruction result image of a person coded as 0021 under a certain acquisition strategy; Figure 10 The image shows the reconstruction result of the character code 0021 under the five-segment acquisition strategy; Figure 11 The image shows the reconstruction result of the character code 0021 under the six-segment acquisition strategy; Figure 12 The image shows the reconstruction result of the character code 0072 under the five-segment acquisition strategy; Figure 13 The image shows the reconstruction result of the character code 0150 under the five-segment acquisition strategy; Figure 14 The image shows the reconstruction result of the character code 0178 under the five-segment acquisition strategy; Figure 15 This is a comparison of the local reconstruction results under six acquisition conditions; Figure 16 Comparison of the reconstruction ratio of different segments of a person in a real selfie scene; Figure 17 GT images of people in real selfie scenarios; Figure 18 This is a reconstructed image captured from a real selfie scene. Figure 19 This is a multi-segment reconstruction effect image based on a real selfie scene; Figure 20 A visualization of the standardized human body grid measurement structure for person code 0021; Figure 21 A visualization of the standardized human body grid measurement structure for the person coded as 0178; Figure 22 A schematic diagram showing the marking of ten key points on the upper body of the person coded as 0021; Figure 23 A schematic diagram showing the marking of ten key points on the upper body of the person coded as 0178; Figure 24 A standardized human body grid measurement structure visualization of the real human body; Figure 25 A schematic diagram showing the marking of ten key points on the upper body of a real human body. Detailed Implementation

[0015] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0016] This invention provides a method for 3D human point cloud reconstruction and automatic measurement based on multi-view images, such as... Figure 1 As shown, please follow these steps: Step 1: Using a segmented multi-view acquisition strategy, the human body is divided into multiple parts for image acquisition. Virtual multi-view human body images generated based on the THuman2.0 dataset and real multi-view human body images acquired in real scenes are obtained separately. Both types of data are collectively referred to as multi-view human body images.

[0017] Step 1 specifically involves the following steps: First, the publicly available THuman2.0 dataset is selected as the data source for virtual multi-view human images. The complete human .obj model, along with its materials and textures, is loaded into the Taichi rendering system to generate multi-view human images corresponding to the same human body. Then, the human .obj model is divided into different regions based on body parts, and virtual camera trajectories are planned for each region, enabling the Taichi rendering system to render segmented images of different body parts. In addition to virtual rendering, real multi-view human images are also collected in real-world scenes, such as… Figure 2 As shown, the subject stands on an automatically rotating platform, which rotates 360° at a uniform angular velocity. A liftable camera device records video sequences at different elevation angles for different body parts. After acquisition, the video is parsed into static image frames using ffmpeg to construct a real multi-view image set. Finally, virtual multi-view human images generated based on the THuman2.0 dataset and real multi-view human images collected in real scenes are obtained. Both types of data are collectively referred to as multi-view human images. Figure 3 The data preparation process and segmented rendering results of this invention are illustrated. Taking human samples and their corresponding real-world images from the THuman2.0 dataset as examples, the figures sequentially show: a complete human .obj model, segmented images obtained from different regions of the body, multi-view human images corresponding to the same human body, and real multi-view human images.

[0018] Step 2: Generate corrected multi-view human body images, sparse point clouds and camera parameters and convert their formats. Then, perform densification processing on the corrected multi-view human body images, sparse point clouds and camera parameters after format conversion, and combine semantic segmentation mask to suppress background interference and generate high-resolution human body point cloud P. Step 2: The COLMAP sparse reconstruction algorithm is used to perform geometric correction and sparse reconstruction on the multi-view human images obtained in Step 1, generating corrected multi-view human images, sparse point clouds, and camera intrinsic and extrinsic parameter data. The corrected multi-view human images, sparse point clouds, and camera parameter data are then converted into an input format acceptable to the deep learning 3D reconstruction network MDF-Net model. The converted corrected multi-view human images, sparse point clouds, and camera parameters are then input into the deep learning 3D reconstruction network MDF-Net model. Using the geometric prior provided by the sparse point cloud to assist in depth estimation, multi-scale feature extraction and cross-view fusion are first performed on the corrected multi-view images to generate a coarse depth map. Then, the depth range is corrected by Gaussian and the depth accuracy is refined by Laplacian. Next, the depth map resolution is improved by lightweight super-resolution upsampling to obtain high-precision depth maps for each view. Finally, 3D point cloud back projection is performed based on the high-precision depth maps for each view and the camera parameters. In the process of point cloud fusion combined with multi-view geometric consistency verification, semantic segmentation masks are introduced to suppress background interference, remove redundancy, conflict points, and background noise, and finally complete the densification process to generate a high-resolution human point cloud.

[0019] Step 2 shall be implemented in accordance with the following: Step 2.1 uses the incremental structure of the COLMAP sparse reconstruction algorithm to generate corrected multi-view human images, camera parameter files, and sparse point cloud files from the motion method, specifically: Step 2.1.1: Perform feature extraction and feature matching on the multi-view human images processed in Step 1 to obtain matching feature point pairs for each multi-view human image pair; then select initial image pairs from the multi-view human image pairs, selecting image pairs with a single image pair matching feature point pair number ≥ 1500 and a valid feature point pair number ≥ 800 after removing mismatches, and with the camera baseline in the range of 0.3m~1.5m. Finally, determine the multi-view human image pairs that meet the requirements as initial image pairs; without given camera intrinsic parameters, estimate the camera intrinsic parameter matrix using the simplified Zhang calibration method based on the matching feature point pairs of the initial image pairs, and obtain the focal length, principal point coordinates, and lens distortion parameters. The camera intrinsic parameter matrix is ​​expressed as follows: ,in Focal length , The principal point position is determined in pixel coordinates; subsequently, a lens distortion model is constructed based on the camera intrinsic parameter matrix, and geometric correction is performed on the multi-view human images processed in step 1 to eliminate image distortion, outputting the geometrically corrected multi-view human images; the fundamental matrix is ​​obtained through feature matching. and using the formula Will Convert to ,in For the camera intrinsic parameter matrix, Transpose it. Based on the matrix, For the essential matrix; for the essential matrix Singular value decomposition (SVD) was performed to obtain four sets of candidate camera pose solutions, each of which fully includes the corresponding rotation matrix. Translation vector Subsequently, triangulation is performed on the matching feature point pairs of the initial image pair. The only feasible camera pose solution is selected based on the reprojection error; specifically, the pose solution with an average reprojection error ≤ 2.0 pixels and a reprojection error greater than 2.0 pixels accounting for no more than 5% of the feature point pairs is chosen as the only feasible solution. After solving, the first image in the initial image pair is set as the origin of the world coordinate system, the pose of the second image in the initial image pair is determined, and a projection matrix is ​​constructed. ,in Used to convert homogeneous three-dimensional coordinates Projecting onto the image coordinate system generates homogeneous pixel coordinates The projection relationship is expressed as The projection matrix constructed using the second image in the initial image pair. The homogeneous 3D coordinates X are projected onto the image coordinate system, and the initial sparse point cloud is triangulated to recover the initial sparse point cloud. Abnormal 3D points with reprojection errors greater than 2.0 pixels in the initial sparse point cloud are removed. Step 2.1.2: Add new images one by one based on the initial sparse point cloud. When the number of matching points between the new image and the current point cloud formed by iterative expansion of the initial sparse point cloud is no less than six, use the EPnP algorithm combined with RANSAC to eliminate outliers and estimate camera extrinsic parameters. For matching feature point pairs that are successfully matched with the new image and the current point cloud but have not yet generated 3D points, perform triangulation and eliminate outliers according to constraints, and expand the point cloud and camera pose set. After all multi-view human images are added, use global bundle adjustment combined with the sparse Levenberg-Marquardt algorithm and Schur complement decomposition to optimize the camera pose and 3D point coordinates, eliminate unqualified observation relationships and outlier 3D points, and finally output the corrected multi-view human images, camera parameter files, and sparse point cloud files using the COLMAP sparse reconstruction algorithm.

[0020] Step 2.2: Combining the corrected multi-view human images, sparse point cloud files and camera parameter files output in Step 2.1, the MDF-Net deep learning 3D reconstruction network model is used to achieve dense reconstruction of multi-view stereo vision. like Figure 4 As shown, the deep learning 3D reconstruction network MDF-Net model consists of four functional modules: a 2D feature pyramid network module, a deep hypothesis and cost volume construction module, a probabilistic inference module, and a super-resolution network module. The specific structures of the four modules are as follows: The 2D feature pyramid network module adopts a shared weight convolutional backbone structure, divided into three feature extraction levels from top to bottom. The first level outputs a feature map with 64 channels and a resolution of 1 / 8 of the input image through 4 convolutions and a stride of 2 downsampling. The second level outputs a feature map with 32 channels and a resolution of 1 / 4 of the input image through 2 convolutions and a stride of 2 downsampling. The third level directly outputs a feature map with 16 channels and a resolution of 1 / 2 of the input image. The fusion structure adopts a top-down feature upsampling + lateral skip connection mode. The upsampling uses bilinear interpolation, and the lateral connections are uniformly added to the corresponding level feature map element by element after a 1×1 convolution to unify the number of channels. The depth hypothesis and cost volume construction module is a three-stage cascaded structure, consisting of a coarse depth estimation submodule, a Gaussian correction submodule, and a Laplacian refinement submodule. Each stage contains a three-level substructure of "feature projection - cost calculation - cost aggregation". Feature projection uses differentiable homography transformation to achieve feature distortion from the source view to the reference view. Cost calculation calculates the feature inner product between the reference view and the source view after performing softmax normalization on the grouped features. Cost aggregation concatenates the costs of all source views by channel, and finally outputs a 4-dimensional cost volume with the dimensions of "number of feature groups G × number of depth samples D × feature map height H × feature map width W". The probabilistic volume inference module is based on a 3D UNet symmetric architecture and is divided into an encoder and a decoder. The encoder contains four 3D convolutional blocks, each consisting of two 3×3×3 convolutional layers, a batch normalization layer, and a ReLU activation function layer. A 2×2×2 stride downsampling is performed after each convolutional block. The decoder is symmetrical to the encoder. Each decoder unit consists of one 2×2×2 transposed convolutional layer, a concatenation operation of the feature maps of the corresponding layer of the encoder, two 3×3×3 convolutional layers, a batch normalization layer, and a ReLU activation function layer. The output is a probabilistic volume with the same size as the input cost volume. The super-resolution network module is based on the EDSR residual network architecture. The input layer is a 1×1 convolutional layer, and the intermediate layer consists of 6 consecutive residual blocks stacked together. Each residual block contains 2 layers of 3×3 convolutional layers + batch normalization layer + ReLU activation function layer. Shortcut skip connections are used inside the residual block. The output layer uses the PixelShuffle upsampling module (magnification factor of 2) and 1 layer of 3×3 convolutional layer to adjust the number of output channels. The final output is a feature map with the same resolution as the original input image. The dense reconstruction process of the deep learning 3D reconstruction network MDF-Net model is as follows: Step 2.2.1: Select one image from the corrected multi-view human images output in Step 2.1 as the reference view, and the rest as source views. Use a 2D feature pyramid network with shared weights to extract three sets of multi-scale features. The feature map channels are 64, 32, and 16, corresponding to resolutions of 1 / 8, 1 / 4, and 1 / 2 of the input image, respectively. These features are used in the four stages of coarse depth estimation, Gaussian correction, Laplacian refinement, and super-resolution. Based on the camera parameter file output in Step 2.1, in the four stages of coarse depth estimation, Gaussian correction, Laplacian refinement, and super-resolution, the source view features are projected onto the reference view using differentiable homography and bilinear interpolation based on the assumed depth. The projection points are calculated using the following formula: (1) In the formula, , , These are the camera intrinsic parameters, rotation matrix, and translation vector of the reference view, respectively. , , The first The corresponding parameters of each source view. Let be the homogeneous coordinates after projection. After completing the projection alignment of the source view features to the reference view, the aligned features of the reference view and all source views are uniformly divided into G groups according to the number of channels. First, the features of each group are normalized by softmax, and then the cost of the reference view and each source view is calculated for each group. The set of costs for all groups constitutes the first set. The cost body corresponding to each source view ; Step 2.2.2: The VCR-Net network then generates the weights for each source view. The VCR-Net network is a two-layer 3D CNN structure with 1×1 convolutional kernels. The specific structure is as follows: The input is a feature volume of dimension G×D×H×W, where G is the number of feature groups, D is the number of depth samples, and H and W are the height and width of the feature map, respectively. The first layer performs a 3D CNN convolution operation with a kernel size of 1×1×1 and a stride of 1. After convolution, it is processed by the ReLU activation function and batch normalized, outputting a feature volume of dimension 1×D×H×W. The second layer also performs a 3D CNN convolution operation with a kernel size of 1×1×1 and a stride of 1. After convolution, it is processed by the Sigmoid activation function without batch normalization, finally outputting a probability volume of dimension 1×D×H×W. The 1×D×H×W probability volumes of each source view are used as the weights for the second layer. Weight of each source view The cost volumes corresponding to all source views are weighted and aggregated to form the final cost volume. : (2) In the formula, Indicates the pixel position of the reference view. The ultimate cost, For the first The cost body corresponding to each source view. For the first The weights obtained after processing the source view costs using VCR-Net Total number of source views; Step 2.2.3: The three stages of coarse depth estimation, Gaussian correction, and Laplacian refinement all employ 3D UNet to regularize the cost volumes generated in each stage. The probability volumes for each stage are obtained using the softmax function, and the depth maps for each stage are obtained through soft argmin regression. Specifically, the coarse depth estimation stage combines the sparse point cloud file output from Step 2.1 to statistically analyze the depth distribution range of the sparse point cloud to verify and correct the preset maximum depth range [425.0 mm, 935.0 mm]. Within the corrected depth range, 48 depth planes are uniformly sampled to obtain the initial depth map. The Gaussian correction stage performs Gaussian distribution fitting based on the probability volume output from the coarse depth estimation stage, with the target curve being: (3) In the formula, As a deep hypothesis, This represents the probability volume value corresponding to the depth hypothesis. The mean parameter of the Gaussian distribution. The standard deviation parameter is the Gaussian distribution parameter; the new depth refinement interval is determined based on the fitted target curve. ,in To estimate the stage regression depth value for the coarse depth map, For the fitting parameters, The probability threshold is used, and 24 depth planes are uniformly sampled within the depth refinement interval; the Laplace refinement stage fits the Laplace distribution based on the probability volume output from the Gaussian correction stage, and the target curve for fitting is: (4) In the formula, It is a depth hypothesis The corresponding probability value, The depth value obtained by Gaussian-corrected regression. This is the scale parameter of the Laplacian distribution. The parameter is also solved using logarithmic transformation and least squares, and a narrower depth interval is used to obtain a high-precision depth map; the refinement interval can be as small as 1mm-3mm. In the super-resolution stage, instead of constructing a cost volume, a lightweight super-resolution network based on the EDSR (Enhanced Deep Residual Network, a classic residual network in the field of single-image super-resolution) structure is used to upsample the high-precision depth map output from the Laplacian refinement stage to the original resolution. A high-resolution depth map is then reconstructed through convolution, residual blocks, and pixel shuffle. Step 2.2.4: Introduce a human semantic segmentation module into the deep learning 3D reconstruction network MDF-Net model to generate a high-precision human binary segmentation mask. Specifically: First, the human semantic segmentation module performs channel normalization on the corrected multi-view human images: grayscale images are converted to BGR format, PNG images are converted to BGR format after removing the alpha channel, and JPG / JPEG images are converted to RGB format. Then, the normalized images are converted to PIL format and input into the rembg network to extract the human foreground, outputting an RGBA image containing the alpha channel, where white represents the foreground and black represents the background. By extracting the alpha channel of this RGBA image and converting it to a PyTorch tensor, the pixel values ​​are binarized to generate a human binary segmentation mask. The definition is as follows: (5) In the formula, Simultaneously for pixels Human body binary segmentation mask value, The pixel position is used as a reference in the view; this is combined with a photometric consistency mask. With geometric consistency mask Perform pixel Joint screening, photometric consistency mask The definition is as follows: (6) In the formula, For the pixel in the i-th view The intensity value, Let be the intensity value of the corresponding pixel in the j-th view. For projection function, For pixels The corresponding three-dimensional point coordinates Preset photometric consistency threshold; geometric consistency mask The definition is as follows: (7) In the formula, For the pixel in the i-th view The intensity value, Let be the depth value of the corresponding pixel in the j-th view. For projection function, For pixels The corresponding three-dimensional point coordinates A preset geometric consistency threshold is set; photometric consistency mask is used. Geometric consistency mask Human body binary segmentation mask By applying joint constraints, the optimized pixel validity mask is obtained, as shown in the following formula: (8) In the formula, For pixels The final validity mask value, " is an element-wise multiplication operation; finally, it is fused to generate a dense human point cloud P, and the 3D reconstruction result is output in .ply format.

[0021] Step 3: Generate the initial SMPL-X human body mesh M 0 Based on the parameters, and combined with the dense human point cloud P generated in step 2, a non-standard human pose mesh M* is obtained through optimization. The non-standard human pose mesh M* is then subjected to pose normalization processing to output a normalized pose human mesh. and corresponding parameters; Step 3: Using the PyMAF tool, perform SMPL-X initial fitting on the corrected multi-view human image output in Step 2.1 to obtain an initial human mesh M that matches the image. 0 ; Set the initial human body mesh M 0 The dense human point cloud P generated in step 2 is coarsely aligned with the mesh. The mesh shape is iteratively adjusted through multi-stage layering and multi-constraint joint optimization to obtain a fitted non-standard human pose mesh M*. Finally, the pose parameters of the non-standard human pose mesh M* are extracted, T-pose standard parameters are constructed, pose transformation is completed using a linear hybrid skinning algorithm, and after local smoothing, a standardized pose human mesh is output. and the corresponding model parameters.

[0022] Step 3 is implemented as follows: Step 3.1, firstly, use the PyMAF tool to perform initial SMPL-X fitting on the corrected multi-view human image output in Step 2.1. The PyMAF tool can quickly regress the shape parameters of the SMPL-X model based on the frontal view of a single corrected multi-view human image. Attitude parameters Translation parameters and the standard SMPL-X initial human body mesh M 0 The fitting process then minimizes the initial human body mesh M of SMPL-X. 0 The L1 loss parameters for iteratively optimizing the rendered human body contour and image contour are as follows: (9) In the formula, For L1 loss, Initial human body mesh M for SMPL-X 0 The rendered human body outline For the image contour, iterate 3-5 times until the loss converges, and finally output the SMPL-X initial human mesh M that matches the pose and shape of the input human image. 0 ; Step 3.2, using the SMPL-X initial human body mesh M generated in step 3.1 0 Based on this, and combined with the dense human point cloud P generated in step 2, the initial human mesh M of SMPL-X is jointly optimized through multi-stage layering and multi-constraint optimization. 0 The details are as follows: First, create the initial human body mesh M using SMPL-X. 0 A rigid transformation is used to align the center of the mesh with the center of the dense human point cloud P, achieving complete global alignment. Then, an iterative nearest-point algorithm is introduced for fine-grained coarse alignment, using Chamfer Distance as the alignment metric. The rotation and translation parameters of the mesh are iteratively optimized to ensure the initial human mesh M... 0 The global pose deviation from the dense human point cloud P is less than 5°, providing high-quality initial values ​​for subsequent fine-tuning optimization. The shape parameters of the SMPL-X model... Attitude parameters Translation parameters To optimize the variables, a multi-constraint optimization objective is constructed, including geometric alignment, topological consistency, and human prior. The specific constraints are as follows: Chamfer Distance is used to measure the initial human grid M. 0 The difference between vertices and the dense point cloud P is expressed as: (10) In the formula, For Chamfer Distance loss, For grid vertices, For points in a point cloud, It is the number of grid vertices. This refers to the number of points in the point cloud. Then, a normal consistency constraint is introduced. This is used to constrain the directional consistency between the vertex normals of the human body mesh and the normals of the dense point cloud. The normal consistency loss is: (11) In the formula, Let be the normal vector of the grid vertex. Let be the normal vector of the point cloud. The pose prior constraint uses the joint rotation angle as a variable to impose kinematic restrictions on each joint; its loss form is defined as: (12) In the formula, For attitude prior loss, Let K be the rotational parameter of the k-th joint, where K is the number of joints. Here is the joint angle penalty function. Prior shape constraints are applied to ensure the shape parameters reside in a reasonable shape space; their loss is defined as: (13) In the formula, For the prior loss of form, This is the shape parameter vector of SMPL-X. Its norm is 2; the local Laplacian smoothing term is used to normalize the surface variation of the mesh in the local neighborhood, and its loss is defined as: (14) In the formula, For local Laplacian smoothing loss, For the i-th grid vertex, Let j be the set of vertices in the neighborhood of vertex i, and j be the set of vertices in the neighborhood. any vertex index in the array, Let be the coordinates of the j-th vertex in the neighborhood; the mesh self-intersection penalty term is used to detect and eliminate the interpenetration relationship between triangular faces that occurs during the optimization process, and its loss is defined as: (15) In the formula, For grid self-intersection penalty loss, and For any two distinct triangular facets, This indicates a pair of facets that intersect or cross each other. This is the self-intersection penalty function. Combining all constraints, we obtain the overall objective function. : (16) In the formula, It is the weight of Chamfer Distance. It is the weight of the normal consistency loss. These are the weights of the attitude prior constraints. These are the weights of the prior constraints on the shape. These are the weights of the local Laplace smoothing constraint. These are the weights of the mesh self-crossing penalty loss. The mesh parameters are iteratively updated using the Adam optimizer. The update rules are as follows: (17) In the formula, For learning rate, It is the gradient of the loss function with respect to the grid parameters; After iteration, a non-standard human pose mesh M* that fits the shape of the dense human point cloud P is obtained; Step 3.3: Convert the fitted non-standard human pose mesh M* into a standard T-pose, as follows: Extracting pose parameters from a non-standard human pose mesh M* Separate the global pose and local joint pose, and construct the standard pose parameters for the T-pose. The root joint rotation is set to an identity matrix. Both arm joints are rotated to a horizontally extended position, and both leg joints are rotated to a straight and joined position. The rotations of the remaining joints are set to a neutral position. Then, based on the SMPL-X skin weights, a linear hybrid skinning algorithm is used to adjust the standard pose parameters. The formula applied to the fitted shape mesh is as follows: (18) In the formula, Let be the skinning weight of joint j corresponding to vertex i. for The rotation matrix of joint j. Let J be the position of joint j in M*. To determine the standard position of joint j in pose T, the transformed pose mesh is locally smoothed to remove vertex overlap or surface wrinkles that may occur during pose transformation, resulting in the final standardized pose human mesh. and its corresponding model parameters.

[0023] Step 4: Based on the standardized pose human mesh obtained in Step 3 The algorithm automatically extracts key human body size indicators and determines key landmarks for clothing fit analysis, based on the corresponding model parameters and the algorithm. This completes automated human body measurement and outputs structured files and visualization results.

[0024] Step 4 is implemented as follows: First, load the standardized pose human body mesh generated in Step 3 according to the automatic measurement algorithm defined by the key points provided by SMPL-X. The model parameters and the difference between the maximum and minimum values ​​of the grid vertices on the Y-axis of the world coordinate system are used to obtain the height. Then, key dimensions are measured: based on the model parameters, the 3D coordinates of 24 joint points are calculated, and then the linear dimensions of shoulder width, left and right arm length, and left and right leg length are measured. For circumference measurement, the cross-sectional search range is first located by joint points, and the hip circumference cross-section with the largest area is found by sliding the virtual horizontal plane. Combined with the spinal joint points, the chest and waist circumference cross-sections are determined. The vertices of each cross-section are extracted and projected onto the XZ plane. The convex hull algorithm is used to calculate the perimeter of the cross-section, completing the chest, waist, and hip circumference measurements. Simultaneously, ten key landmark points are dynamically generated according to the definition of clothing measurement, including six feature points on the left and right shoulders, left and right hips, and chest and waist cross-sections. Through initial joint point positioning, finding grid intersections by ray projection along the Z-axis, cross-sectional geometric screening, symmetry mirroring, and linear interpolation, the landmark points are ensured to be located on the human body surface and satisfy symmetry. Finally, the key dimension indicators and the 3D coordinates of the landmark points are saved as JSON or TXT structured files, generating two visualization charts: one showing the human body grid, measurement cross-sections, and joint points, and the other highlighting the ten key landmark points.

[0025] Example 1 Case Study and Method Validation: This embodiment is used to verify the effectiveness of the improved dense reconstruction method. The hardware equipment and environmental parameters used for experimental training are shown in Table 1. Table 1 Experimental hardware equipment and environmental parameters

[0026] To verify the improved effects of the method of this invention in terms of background suppression, geometric consistency, and structural integrity, this embodiment conducted experimental verification and performance comparison from four aspects: mask processing, point cloud sampling density, number of single-loop views, and segmented acquisition strategy. Figure 5 The image shown is the reconstruction result of a dense point cloud without masking. It can be seen that the reconstruction without masking exhibits significant background noise and structural inconsistencies, especially in the head, limb edges, and surrounding areas, resulting in numerous discrete points or pseudo-contours, leading to blurred model boundaries and structural discontinuities. However, after masking, as shown... Figure 6 As shown, the human body region is effectively extracted, background interference is significantly reduced, the point cloud contour boundaries are clearer, and the overall structural coherence is significantly improved. This demonstrates that the masking mechanism can significantly enhance the network's ability to perceive the foreground human body, improve photometric consistency and geometric constraints, and provide a more stable input basis for subsequent measurement steps.

[0027] Based on the optimization of masking, to balance the point cloud reconstruction quality and subsequent measurement efficiency, a random human sample was selected for accuracy evaluation under different sampling densities. Specifically, the reconstructed point cloud was uniformly sampled at 20,000, 50,000, 100,000, 150,000, and 200,000 points, and the error indices between it and the real human mesh were calculated, including Chamfer Distance (CD), Normal Consistency (NC), and Point-to-Surface Distance (P2S). As shown in Table 2, with the increase of the number of sampling points, the CD and P2S indices gradually decreased, and the geometric accuracy improved significantly; while the NC value increased slightly under high-density point clouds, mainly due to local normal fluctuations. Considering factors such as reconstruction fidelity, stability, efficiency, and computational resource consumption, a sampling density of 100,000 points was determined to achieve the best balance, and this configuration was adopted in subsequent experiments.

[0028] Table 2. Impact of Sampling Density Variation on Reconstruction Accuracy

[0029] Furthermore, to evaluate the impact of single-loop viewpoint acquisition configuration on reconstruction performance, six different viewpoint configurations were set up on the same human sample, with 30, 36, 40, 48, 56, and 60 evenly distributed camera positions, to simulate traditional single-loop multi-view shooting methods. Figure 7 As shown in the figure, (a)–(f) sequentially illustrate the reconstruction results under the above six viewpoint configurations. As can be seen from the figure, as the number of viewpoints increases, the overall outline of the model gradually becomes more complete. However, when the viewpoints are too dense, excessive angular overlap will introduce redundant observations and inconsistencies in depth fusion, resulting in noise and surface blurring in local areas. On the other hand, if the number of viewpoints is too small, it will cause the point cloud of the limbs and torso to be missing, resulting in structural discontinuity.

[0030] To further quantify the impact of different viewpoint configurations on reconstruction accuracy, three indices—CD, NC, and P2S—were calculated for each experiment, and the results are shown in Table 3. The data in the table show that simply increasing the number of viewpoints in a single ring does not significantly improve reconstruction fidelity; on the contrary, inconsistencies in normals and surface fluctuations occur in some high-density configurations. This indicates that single-ring acquisition still has inherent limitations in terms of spatial coverage and geometric constraints, making it difficult to achieve complete and high-precision reconstruction of the human body structure.

[0031] Table 3 Evaluation results corresponding to different number of perspectives

[0032] Based on previous experiments, in order to further verify the effect of the segmented acquisition strategy on improving the accuracy of 3D human body reconstruction, this embodiment selects human body sample number 0021 from the THuman2.0 dataset and conducts comparative experiments under three configurations: single-segment acquisition, five-segment acquisition, and six-segment acquisition. Figure 9 , Figure 10 and Figure 11 Reconstruction results from multiple viewing angles using three different acquisition methods are presented. Figure 8 This serves as a benchmark for the multi-view truth model of this sample. The experimental results show that... Figure 9 The single-segment acquisition method shown exhibits problems such as blurriness, fragmentation, and partial loss in areas like the face, chest, and lower limbs, resulting in significant distortion of the overall structure; in contrast, Figure 10 The five-segment acquisition strategy shown utilizes surround view at different heights for coverage, significantly improving local geometric misalignment and making the structure of the face, torso, and key areas more complete and continuous; furthermore, Figure 11 The six-segment acquisition strategy, as shown, supplements the original multi-height surround acquisition with a top-down perspective, fully compensating for areas that are difficult to capture from traditional perspectives, such as the top of the head, neck, and upper body curved surfaces. This results in significant improvements in both overall integrity and detail reproduction. The comparison results are shown in Table 4. Through statistical analysis of the quantitative evaluation results of different acquisition strategies, the five-segment and six-segment acquisition strategies outperform the single-segment acquisition scheme in all three reconstruction quality indices (CD, NC, and P2S). Specifically, the six-segment acquisition strategy reduces the CD index to approximately 0.6114, significantly reducing the average point cloud deviation; the NC index decreases to 14.31°, indicating improved normal consistency and a smoother, more natural model surface; and the P2S index decreases to 0.2806, further validating the high-precision fitting effect of the point cloud to the real human body surface. Therefore, the multi-segment acquisition strategy can significantly enhance detail reconstruction and surface continuity while maintaining overall structural integrity, providing higher geometric accuracy support for 3D human body reconstruction.

[0033] Table 4 Comparison of 3D Reconstruction Quality Evaluation Indicators under Different Acquisition Strategies

[0034] To verify the applicability of the segmented acquisition strategy under different body types and local structural conditions, this embodiment additionally selects three groups of human samples with person numbers 0072, 0150 and 0178 from the THuman2.0 dataset and conducts supplementary experiments under the five-segment acquisition configuration. Figure 12 , Figure 13 and Figure 14 The ground truth (GT) models of three samples and their corresponding reconstruction results are presented respectively. Figure 12Corresponding sample 0072, Figure 13 Corresponding sample 0150, Figure 14 The corresponding sample is 0178. The comparison results of the three sets of samples show that the five-segment acquisition strategy maintains high reconstruction consistency under different body shapes, posture variations, and surface texture differences. The geometric structures of the head, torso, and limbs in each sample are completely presented, with continuous surface boundaries and no obvious breaks in local curved surfaces. It can stably generate 3D human body models with clear contours and high detail, further verifying the universality and robustness of the acquisition strategy on diverse individuals. The corresponding quantitative evaluation results are shown in Table 5. The three sets of samples all show low error levels in the CD, NC, and P2S indices, verifying the stability and accuracy of this acquisition method in overall structural reconstruction.

[0035] Table 5 Comparison of 3D Reconstruction Indicators under Five Segments

[0036] To further observe the improvement effect of six-segment acquisition on local geometric details, such as Figure 15 As shown, the comparison presents the results of upper body and facial reconstruction under five-segment and six-segment acquisition conditions for the same three samples. The local comparison images reveal that six-segment acquisition, by adding a top-down perspective, effectively supplements information in the top-view areas such as the top of the head and neck, resulting in more realistic and smoother details in facial contours, skull tops, and neck-shoulder transitions, significantly improving the overall naturalness of the model.

[0037] The corresponding quantitative evaluation results are shown in Table 6. The six-segment acquisition method further optimizes all indicators compared to the five-segment configuration, with an overall decrease in CD and P2S values, indicating a reduction in average geometric error and more accurate surface fitting. Comparative analysis shows that this strategy exhibits good reconstruction consistency and detail restoration capabilities across different individuals, verifying the robustness and stability of the segmented multi-loop acquisition method under diverse human sample conditions.

[0038] Table 6 Comparison of 3D Reconstruction Indicators under Six Segments

[0039] After completing the aforementioned human reconstruction experiment based on simulated acquisition conditions, to further verify the adaptability and effectiveness of the method of the present invention in real-world scenarios, this embodiment conducts actual tests based on autonomously acquired multi-view human image data. The image data was obtained through multi-view shooting in a real environment, reflecting the influence of real factors such as natural lighting, background interference, and posture changes. Based on this data, two configurations—single-segment acquisition and segmented acquisition—were used for 3D reconstruction to evaluate the performance differences of the method of the present invention in terms of preserving local details and restoring the overall structure. Figure 16As shown, the results of 3D human reconstruction based on multi-view selfie data and a comparison of the body's vertical proportions are presented. The first image is the baseline reconstruction result of a real human body, with a vertical proportion of 1.306. The second image is the reconstruction result obtained by single-segment acquisition, with a proportion of 1.413, which deviates significantly from the true proportions, exhibiting distortions such as an elongated upper body and compressed lower limbs. The third image is the reconstruction model obtained by segmented acquisition, with a proportion of 1.305, highly consistent with the true value. The comparison reveals that single-segment acquisition, due to only completing one circumference at a fixed height, results in a severe lack of vertical perspective, easily causing deformation in areas such as the head, neck, and legs, leading to an overall imbalance in geometric proportions. In contrast, segmented acquisition, by supplementing perspectives at multiple heights, significantly improves spatial coverage, accurately restoring the vertical scale relationship of the human body. The constructed 3D model is closer to the real human body in terms of structural proportions, key part morphology, and overall naturalness.

[0040] To more intuitively demonstrate the 3D reconstruction capabilities of this method under different acquisition methods, this embodiment provides multi-angle views of the original image of the baseline model, the reconstruction result obtained from a single acquisition, and the reconstruction result obtained from a multi-segment acquisition method. Figure 17 The original images of the baseline model from multiple angles are shown as a reference for the real structure and shape. Figure 18 The results of multi-angle reconstruction obtained under single-segment acquisition conditions are shown. Figure 19 The results show multi-angle reconstruction obtained using a multi-segment acquisition method. The comparison reveals that the single-segment acquisition method still suffers from local structural discontinuities or missing details in some side views, while the multi-segment acquisition method maintains the integrity of the model structure, clear outlines, and harmonious geometric proportions across all views within a 360° range. The reconstruction results are closer to the real human form, fully validating the stability and high-fidelity reconstruction capability of this method under complex viewpoint distributions.

[0041] Example 2 Case Study and Method Validation: This embodiment is used to verify the effectiveness and stability of the automatic human body measurement method based on standardized meshes under different body types and complex scenarios. The human body meshes used in the experiment were all obtained by standardizing pose transformation and parametric fitting from dense reconstruction results. The final measurement results were automatically output by the method of this invention without human intervention.

[0042] To verify the performance of the method of the present invention in terms of dimensional accuracy and measurement consistency, two samples with significant differences in body size were selected as comparison subjects. Before measurement, their three-dimensional human body grids were uniformly scaled to a height of about 1.8 meters to eliminate the influence of individual proportion differences. Figure 20The sample shown is a relatively plump one, which has a larger lateral width and fuller volume distribution in the chest, waist and hip areas, and the overall shape shows obvious plump characteristics. The method of the present invention automatically generates horizontal measurement sections of chest, waist and hip circumference under the standardized T-shaped posture of the model, and extracts indicators such as shoulder width, arm length, leg length and height, and visualizes the structural position of each measurement part. Figure 21 The sample shown is a relatively slender body type, exhibiting a more compact cross-sectional shape at key circumferences such as chest, waist, and hips, with a more delicate overall body geometry. The method of this invention automatically identifies the corresponding measurement sites and generates measurement cross-sections in a T-pose, completing the calculation of various anthropometric indicators. Under a unified normalized coordinate system, the measurement cross-section positions of both groups of samples are automatically determined by the method of this invention based on human anatomy: chest circumference at the fourth intercostal plane, waist circumference at the umbilicus plane, and hip circumference at the maximum diameter of the gluteus maximus muscle. Circumference is represented by the perimeter of a closed loop; shoulder width is calculated from the three-dimensional Euclidean distance between the left and right shoulder joints; upper and lower limb lengths are determined by the cumulative distance of the lines connecting the corresponding joints; and height is obtained from the vertical distance from the top of the head to the sole of the foot. The measurement results show that the method of this invention can accurately locate key areas and generate continuous, reasonable measurement structures consistent with body type characteristics. The circumference measurement of the fuller sample is significantly larger than that of the slender sample, fully reflecting the real differences between the two body types and verifying the measurement stability and reliability of the method of this invention under different body type conditions. The automatic measurement results of the two samples are listed in Table 7. The results showed that although the heights of both groups were roughly the same, the fuller-figured sample had significantly higher chest, waist, and hip circumferences than the thinner-figured sample, consistent with visual body shape differences. The differences in shoulder width and limb length were relatively small, indicating that these indicators were mainly influenced by bone proportions. The deviations between the dimensions measured by the method of this invention and manually measured reference values ​​were all controlled within 1 cm, verifying the high accuracy and stability of the automatic measurement algorithm of this invention.

[0043] Table 7. Measurement results of various anthropometric indicators (unit: cm)

[0044] To further verify the effectiveness of the upper body fit assessment module, this embodiment automatically identifies ten key upper body landmarks (LA, RA, LL, RL, V1–V6) on a standardized grid. Figure 22 The spatial distribution of marker points on a fuller-figured sample is shown. LA and RA are located at the bottom of the armpit and are used to define the boundaries of movement of the cuff and shoulder. LL and RL are located at the transition between the lower ribs and the waistline and are used to depict the fit of the garment hem. V1–V3 are distributed on both sides and the midpoint of the spine on the back. V4–V6 are located in the area connecting the clavicle and the armpit on the front of the chest and are used to capture the changes in the front and back torso contours. Figure 23This illustrates the distribution of marker points on relatively thin samples, and the layout and function of these marker points. Figure 22 By maintaining consistency and using unified spatial coordinate calibration, this structure can be used for size mapping, tension distribution, and pattern fitting analysis in clothing design. It also facilitates comparison and quantitative evaluation of fit for samples of different body types. To verify the applicability of the method in complex real-world scenarios, this embodiment uses the dense human point cloud obtained through multi-view image reconstruction in step two as input data for measurement. This point cloud originates from multi-view selfie images taken with a mobile phone, and is affected by changes in lighting, complex backgrounds, and partial occlusion, thus better demonstrating the performance of the method under non-ideal conditions. Figure 24 The SMPL-X visualization model under the standardized T-pose and its seven automatically identified core anthropometric indicators are demonstrated for the location and calculation of chest circumference, waist circumference, hip circumference, shoulder width, arm length, leg length and height, guiding the method of this invention to accurately extract key dimensional information. Figure 25 The model showcases ten key upper body landmarks (LA, RA, LL, RL, V1–V6) used for clothing fit assessment. These landmarks are located under the armpits, at the waistline, and in the front and back torso areas. They are used to analyze garment size mapping, tension distribution, and pattern fitting, while also supporting the comparison and quantitative assessment of fit for samples of different body types.

[0045] To evaluate the repeatability of the method of this invention, five independent measurements were performed on the same subject, and the statistical results are shown in Table 8. The deviations of all indicators measured by the method of this invention from manually measured reference values ​​were all within 1 cm, verifying the high accuracy and reliability of the automatic measurement method. Compared with the standard reference values, the maximum deviation was less than 4 cm, mainly due to the surface projection error of the chest circumference; all other indicators were highly consistent with the true values, verifying the applicability and robustness of this invention in uncontrolled environments.

[0046] Table 8 Comparison of Measurement Results of Various Anthropometric Indicators (Unit: cm)

[0047] In summary, this embodiment verifies that the automatic human body measurement method of the present invention can achieve high-precision and stable three-dimensional dimension extraction under different body types, different acquisition methods, and complex background conditions. This method provides a reliable technical foundation for virtual clothing fit assessment, personalized tailoring, and human body data analysis. This embodiment further demonstrates that the method can achieve high-precision automatic human body measurement in natural light environments without professional equipment, possessing strong practicality and promotional value. The present invention solves the problems of existing technologies, such as the lack of a multi-view reconstruction mechanism oriented towards human structural features, insufficient coverage of acquisition methods, and low accuracy of parametric fitting.

[0048] Example 3 The method for 3D human point cloud reconstruction and automatic measurement based on multi-view images is as follows: Step 1: Acquire virtual multi-view human images and real multi-view human images collected in real scenes; Step 2: Generate calibrated multi-view human images, sparse point clouds, and camera parameters, convert their formats, perform densification processing, and combine semantic segmentation masks to suppress background interference, generating a dense human point cloud P; Step 3: Generate an initial SMPL-X human mesh M. 0 Based on the parameters, and combined with the dense human point cloud P generated in step 2, a non-standard human pose mesh M* is obtained through optimization. The non-standard human pose mesh M* is then subjected to pose normalization processing to output a normalized pose human mesh. And corresponding parameters; Step 4, complete the automated human body measurement and output structured files and visualization results.

[0049] Example 4 The difference between Example 4 and Example 3 is that in step 1, the publicly available THuman2.0 dataset is first selected as the data source for virtual multi-view human images.

[0050] Example 5 The difference between Example 5 and Example 4 is as follows: In step 1, the complete human body .obj model and its materials and textures are loaded into the Taichi rendering system to generate multi-view human body images corresponding to the same human body. Then, the human body .obj model is divided into different regions of the body parts, and virtual camera trajectories are planned for each region, so that the Taichi rendering system can render segmented images obtained from different regions of the body parts. In addition to virtual rendering, real multi-view human body images are also collected in real scenes. The subject stands on an automatic rotating platform, which completes a 360° rotation at a uniform angular velocity. A liftable camera device records video sequences at different elevation angles according to different regions of the body parts. After collection, the video is parsed into static image frames using ffmpeg to construct a real multi-view image set. Finally, the virtual multi-view human body images generated based on the THuman2.0 dataset and the real multi-view human body images collected in real scenes are obtained. Both types of data are collectively referred to as multi-view human body images.

[0051] Example 6 Example 6 is as follows: Step 2.1, using the incremental structure of the COLMAP sparse reconstruction algorithm, the corrected multi-view human body image, camera parameter file, and sparse point cloud file are generated from the motion method; Step 2.2, combining the corrected multi-view human body image, sparse point cloud file, and camera parameter file output in Step 2.1, the deep learning 3D reconstruction network MDF-Net model is used to realize multi-view stereo vision dense reconstruction.

Claims

1. A method for 3D human point cloud reconstruction and automatic measurement based on multi-view images, characterized in that, Specifically: Step 1: Acquire virtual multi-view human body images and real multi-view human body images captured in real scenes respectively; Step 2: Generate corrected multi-view human body images, sparse point clouds, and camera parameters, convert their formats, perform densification processing, and combine semantic segmentation masks to suppress background interference, generating dense human body point clouds P; Step 3: Generate the initial SMPL-X human body mesh M 0 Based on the parameters, and combined with the dense human point cloud P generated in step 2, a non-standard human pose mesh M* is obtained through optimization. The non-standard human pose mesh M* is then subjected to pose normalization processing to output a normalized pose human mesh. and corresponding parameters; Step 4: Complete the automated anthropometric measurements and output structured documents and visualization results.

2. The method for three-dimensional human point cloud reconstruction and automatic measurement based on multi-view images according to claim 1, characterized in that, In step 1, the publicly available THuman2.0 dataset is first selected as the data source for virtual multi-view human images. The complete human .obj model and its materials and textures from the dataset are loaded into the Taichi rendering system to generate multi-view human images corresponding to the same human body. Then, the human .obj model is divided into different regions of the body parts, and virtual camera trajectories are planned for each region, enabling the Taichi rendering system to render segmented images of different regions of the body parts. In addition to virtual rendering, real multi-view human images are also collected in real scenes. The subject stands on an automatically rotating platform, which completes a 360° rotation at a uniform angular velocity. A liftable camera device records video sequences at different elevation angles for different regions of the body parts. After collection, the video is parsed into static image frames using ffmpeg to construct a real multi-view image set. Finally, the virtual multi-view human images generated based on the THuman2.0 dataset and the real multi-view human images collected in real scenes are obtained. Both types of data are collectively referred to as multi-view human images.

3. The method for three-dimensional human point cloud reconstruction and automatic measurement based on multi-view images according to claim 2, characterized in that, Step 2 shall be implemented in accordance with the following: Step 2.1: Using the incremental structure of the COLMAP sparse reconstruction algorithm, corrected multi-view human images, camera parameter files, and sparse point cloud files are generated from the motion method. Step 2.2: Combining the corrected multi-view human images, sparse point cloud files and camera parameter files output in Step 2.1, the MDF-Net deep learning 3D reconstruction network model is used to achieve dense reconstruction of multi-view stereo vision.

4. The method for three-dimensional human point cloud reconstruction and automatic measurement based on multi-view images according to claim 3, characterized in that, Step 2.1 specifically involves: Step 2.1.1: Perform feature extraction and feature matching on the multi-view human images processed in Step 1 to obtain matching feature point pairs for each multi-view human image pair; Subsequently, initial image pairs were selected from the multi-view human body image pairs. Image pairs with ≥1500 matching feature point pairs per image pair and ≥800 valid feature point pairs after removing false matches, along with camera baselines within the range of 0.3m to 1.5m, were chosen as the initial image pairs. Without given camera intrinsic parameters, the camera intrinsic parameter matrix was estimated using a simplified Zhang calibration method based on the matching feature point pairs of the initial image pairs. The focal length, principal point coordinates, and lens distortion parameters were then obtained. The camera intrinsic parameter matrix is ​​expressed as follows: ,in Focal length , The principal point position is defined in the pixel coordinate system. Then, a lens distortion model is constructed based on the camera intrinsic parameter matrix. Geometric correction is performed on the multi-view human images processed in step 1 to eliminate image distortion, and the geometrically corrected multi-view human images are output. Obtaining the fundamental matrix through feature matching and using the formula Will Convert to ,in For the camera intrinsic parameter matrix, Transpose it. Based on the matrix, For the essential matrix; for the essential matrix Singular value decomposition yields four sets of candidate camera pose solutions, each of which fully includes the corresponding rotation matrix. Translation vector Then, triangulation is performed on the matching feature point pairs of the initial image pair. The only feasible camera pose solution is selected based on the reprojection error, that is, the pose solution with an average reprojection error ≤ 2.0 pixels and a proportion of feature point pairs with a reprojection error greater than 2.0 pixels does not exceed 5%. After solving the problem, the first image in the initial image pair is set as the origin of the world coordinate system. The pose of the second image in the initial image pair is determined, and the projection matrix is ​​constructed. ,in Used to convert homogeneous three-dimensional coordinates Projecting onto the image coordinate system generates homogeneous pixel coordinates The projection relationship is expressed as ; Projection matrix constructed using the second image in the initial image pair The homogeneous 3D coordinates X are projected onto the image coordinate system, and the initial sparse point cloud is triangulated to recover the initial sparse point cloud. Abnormal 3D points with reprojection errors greater than 2.0 pixels in the initial sparse point cloud are removed. Step 2.1.2: Add new images one by one based on the initial sparse point cloud. When the number of matching points between the new image and the current point cloud formed by iterative expansion of the initial sparse point cloud is no less than six, use the EPnP algorithm combined with RANSAC to eliminate outliers and estimate camera extrinsic parameters. For matching feature point pairs that are successfully matched with the new image and the current point cloud but have not yet generated 3D points, perform triangulation and eliminate outliers according to constraints, and expand the point cloud and camera pose set. After all multi-view human images are added, use global bundle adjustment combined with the sparse Levenberg-Marquardt algorithm and Schur complement decomposition to optimize the camera pose and 3D point coordinates, eliminate unqualified observation relationships and outlier 3D points, and finally output the corrected multi-view human images, camera parameter files, and sparse point cloud files using the COLMAP sparse reconstruction algorithm.

5. The method for three-dimensional human point cloud reconstruction and automatic measurement based on multi-view images according to claim 4, characterized in that, In step 2.2: The deep learning 3D reconstruction network MDF-Net model consists of four functional modules: a 2D feature pyramid network module, a deep hypothesis and cost volume construction module, a probabilistic volume inference module, and a super-resolution network module. The specific structures of the four modules are as follows: The 2D feature pyramid network module adopts a shared weight convolutional backbone structure, divided into three feature extraction levels from top to bottom. The first level outputs a feature map with 64 channels and a resolution of 1 / 8 of the input image through 4 convolutions and downsampling with a stride of 2. The second level outputs a feature map with 32 channels and a resolution of 1 / 4 of the input image through 2 convolutions and downsampling with a stride of 2. The third level directly outputs a feature map with 16 channels and a resolution of 1 / 2 of the input image. The fusion structure adopts a top-down feature upsampling + lateral skip connection mode. The upsampling adopts bilinear interpolation, and the lateral connection is added element-wise to the corresponding level feature map after unifying the number of channels through 1×1 convolution. The depth hypothesis and cost volume construction module is a three-stage cascaded structure, consisting of a coarse depth estimation submodule, a Gaussian correction submodule, and a Laplacian refinement submodule. Each stage has a built-in three-level substructure of "feature projection - cost calculation - cost aggregation". Feature projection uses differentiable homography transformation to achieve feature distortion from the source view to the reference view. Cost calculation calculates the feature inner product between the reference view and the source view after performing softmax normalization on the grouped features. Cost aggregation concatenates the costs of all source views by channel, and finally outputs a 4-dimensional cost volume with the dimensions of "number of feature groups G × number of depth samples D × feature map height H × feature map width W". The probabilistic volume inference module is based on the 3D UNet symmetric architecture and is divided into an encoder and a decoder. The encoder contains four 3D convolutional blocks, each consisting of two 3×3×3 convolutional layers, a batch normalization layer, and a ReLU activation function layer. A 2×2×2 stride downsampling is performed after each convolutional block. The decoder is symmetrical to the encoder. Each decoder unit consists of one 2×2×2 transposed convolutional layer, a concatenation operation of the feature maps of the corresponding layer of the encoder, two 3×3×3 convolutional layers, a batch normalization layer, and a ReLU activation function layer. The output is a probabilistic volume with the same size as the input cost volume. The super-resolution network module is based on the EDSR residual network architecture. The input layer is a 1×1 convolutional layer, and the intermediate layer consists of 6 consecutive residual blocks stacked together. Each residual block contains 2 layers of 3×3 convolutional layers + batch normalization layer + ReLU activation function layer. Shortcut skip connections are used inside the residual block. The output layer uses the PixelShuffle upsampling module, combined with a 3×3 convolutional layer to adjust the number of output channels. The final output is a feature map with the same resolution as the original input image.

6. The method for three-dimensional human point cloud reconstruction and automatic measurement based on multi-view images according to claim 4, characterized in that, Step 2.2 specifically involves: Step 2.2.1: Select one of the corrected multi-view human images output from Step 2.1 as the reference view, and the rest as the source views. Use a 2D feature pyramid network with shared weights to extract three sets of multi-scale features. The number of channels in the feature maps are 64, 32 and 16, respectively, and the corresponding resolutions are 1 / 8, 1 / 4 and 1 / 2 of the input image, respectively. These features are used for four stages: coarse depth estimation, Gaussian correction, Laplacian refinement and super-resolution. Based on the camera parameter file output in step 2.1, in the four stages of coarse depth estimation, Gaussian correction, Laplacian refinement, and super-resolution, the source view features are projected onto the reference view using differentiable homography and bilinear interpolation according to the assumed depth. The projection points are calculated by the following formula: （1） In the formula, , , These are the camera intrinsic parameters, rotation matrix, and translation vector of the reference view, respectively. , , The first The corresponding parameters of each source view. These are the homogeneous coordinates after projection; After completing the projection alignment of the source view features to the reference view, the aligned features of the reference view and all source views are uniformly divided into G groups according to the number of channels. Each group of features is first normalized using softmax, and then the cost of the reference view and each source view is calculated for each group. The set of costs for all groups constitutes the first G group. The cost body corresponding to each source view ; Step 2.2.2: The VCR-Net network then generates the weights for each source view. The VCR-Net network is a two-layer 3D CNN structure with 1×1 convolutional kernels. The specific structure is as follows: The input is a feature volume of G×D×H×W dimensions, where G is the number of feature groups, D is the number of depth samples, and H and W are the height and width of the feature map, respectively. The first layer performs 3D CNN convolution operation with a kernel size of 1×1×1 and a stride of 1. After convolution, it is processed by the ReLU activation function and batch normalized. The output is a feature volume of 1×D×H×W dimensions. The second layer also performs 3D CNN convolution operations, with the kernel size remaining 1×1×1 and a stride of 1. After convolution, it is processed by the Sigmoid activation function without batch normalization, ultimately outputting a probability volume of 1×D×H×W dimensions. The 1×D×H×W probability volume of each source view is used as the first... Weight of each source view The cost volumes corresponding to all source views are weighted and aggregated to form the final cost volume. : （2） In the formula, Indicates the pixel position of the reference view. The ultimate cost, For the first The cost body corresponding to each source view. For the first The weights obtained after processing the source view costs using VCR-Net Total number of source views; Step 2.2.3: The three stages of coarse depth estimation, Gaussian correction, and Laplacian refinement all employ 3D UNet to regularize the cost volumes generated at each stage. The probability volumes for each stage are obtained using the softmax function, and the depth maps for each stage are obtained through softargmin regression. Specifically, the coarse depth estimation stage combines the sparse point cloud file output from Step 2.1 to statistically analyze the depth distribution range of the sparse point cloud to verify and correct the preset maximum depth range [425.0 mm, 935.0 mm]. Within the corrected depth range, 48 depth planes are uniformly sampled to obtain the initial depth map. The Gaussian correction stage performs Gaussian distribution fitting based on the probability volume output from the coarse depth estimation stage, with the target curve being: （3） In the formula, As a deep hypothesis, This represents the probability volume value corresponding to the depth hypothesis. The mean parameter of the Gaussian distribution. The standard deviation parameter is the Gaussian distribution parameter; the new depth refinement interval is determined based on the fitted target curve. ,in To estimate the stage regression depth value for the coarse depth map, For the fitting parameters, The probability threshold is used, and 24 depth planes are uniformly sampled within the depth refinement interval; the Laplace refinement stage fits the Laplace distribution based on the probability volume output from the Gaussian correction stage, and the target curve for fitting is: （4） In the formula, It is a depth hypothesis The corresponding probability value, The depth value obtained by Gaussian-corrected regression. The scale parameter of the Laplacian distribution is used; the parameter is also solved by logarithmic transformation and least squares method, and a narrower depth interval is used to obtain a high-precision depth map, which can be as small as 1mm-3mm; the cost volume is no longer constructed in the super-resolution stage, but a lightweight super-resolution network based on EDSR structure is used to upsample the high-precision depth map output by the Laplacian refinement stage to the original resolution, and a high-resolution depth map is reconstructed through convolution, residual block and PixelShuffle. Step 2.2.4: Introduce a human semantic segmentation module into the deep learning 3D reconstruction network MDF-Net model to generate a high-precision human binary segmentation mask. Specifically: First, the human semantic segmentation module performs channel normalization on the corrected multi-view human images: grayscale images are converted to BGR format, PNG images are converted to BGR format after removing the alpha channel, and JPG / JPEG images are converted to RGB format. Then, the normalized images are converted to PIL format and input into the rembg network to extract the human foreground, outputting an RGBA image containing the alpha channel, where white represents the foreground and black represents the background. By extracting the alpha channel of this RGBA image and converting it to a PyTorch tensor, the pixel values ​​are binarized to generate a human binary segmentation mask. The definition is as follows: （5） In the formula, Simultaneously for pixels Human body binary segmentation mask value, The pixel position is used as a reference in the view; this is combined with a photometric consistency mask. With geometric consistency mask Perform pixel Joint screening, photometric consistency mask The definition is as follows: （6） In the formula, For the pixel in the i-th view The intensity value, Let be the intensity value of the corresponding pixel in the j-th view. For projection function, For pixels The corresponding three-dimensional point coordinates Preset photometric consistency threshold; geometric consistency mask The definition is as follows: （7） In the formula, For the pixel in the i-th view The intensity value, Let be the depth value of the corresponding pixel in the j-th view. For projection function, For pixels The corresponding three-dimensional point coordinates A preset geometric consistency threshold is set; photometric consistency mask is used. Geometric consistency mask Human body binary segmentation mask By applying joint constraints, the optimized pixel validity mask is obtained, as shown in the following formula: （8） In the formula, For pixels The final validity mask value, " is an element-wise multiplication operation; finally, it is fused to generate a dense human point cloud P, and the 3D reconstruction result is output in .ply format.

7. The method for three-dimensional human point cloud reconstruction and automatic measurement based on multi-view images according to claim 6, characterized in that, Step 3 specifically involves: Step 3.1: First, use the PyMAF tool to perform initial SMPL-X fitting on the corrected multi-view human images output in Step 2.

1. The PyMAF tool can quickly regress the shape parameters of the SMPL-X model based on a single corrected multi-view human image's frontal view. Attitude parameters Translation parameters and the standard SMPL-X initial human body mesh M 0 ; The fitting process then minimizes the initial human body mesh M of SMPL-X. 0 The L1 loss parameters for iteratively optimizing the rendered human body contour and image contour are as follows: （9） In the formula, For L1 loss, Initial human body mesh M for SMPL-X 0 The rendered human body outline For the image contour, iterate 3-5 times until the loss converges, and finally output the SMPL-X initial human mesh M that matches the pose and shape of the input human image. 0 ; Step 3.2, using the SMPL-X initial human body mesh M generated in step 3.1 0 Based on this, and combined with the dense human point cloud P generated in step 2, the initial human mesh M of SMPL-X is jointly optimized through multi-stage layering and multi-constraint optimization. 0 The details are as follows: First, create the initial human body mesh M using SMPL-X. 0 By using rigid transformation, the center of the mesh is made to coincide with the center point of the dense human body point cloud P, achieving complete global position alignment. Then, an iterative nearest-point algorithm is introduced for fine-grained coarse alignment. Using Chamfer Distance as the alignment metric, the rotation and translation parameters of the mesh are iteratively optimized to make the initial human body mesh M... 0 The global pose deviation from the dense human point cloud P is less than 5°; based on the shape parameters of the SMPL-X model Attitude parameters Translation parameters To optimize the variables, a multi-constraint optimization objective is constructed, including geometric alignment, topological consistency, and human prior. The specific constraints are as follows: Chamfer Distance is used to measure the initial human grid M. 0 The difference between vertices and the dense point cloud P is expressed as: （10） In the formula, For Chamfer Distance loss, For grid vertices, For points in a point cloud, It is the number of grid vertices. It is the number of points in the point cloud; then, a normal consistency constraint is introduced. This is used to constrain the directional consistency between the vertex normals of the human body mesh and the normals of the dense point cloud. The normal consistency loss is: （11） In the formula, Let be the normal vector of the grid vertex. Let be the normal vector of the point cloud; the attitude prior constraint uses the joint rotation angle as a variable to impose kinematic restrictions on each joint, and its loss form is defined as: （12） In the formula, For attitude prior loss, Let K be the rotational parameter of the k-th joint, where K is the number of joints. The joint angle penalty function is used; the shape prior constraint is used to ensure that the shape parameters are located in a reasonable shape space, and its loss is defined as: （13） In the formula, For the prior loss of form, This is the shape parameter vector of SMPL-X. Its norm is 2; the local Laplacian smoothing term is used to normalize the surface variation of the mesh in the local neighborhood, and its loss is defined as: （14） In the formula, For local Laplacian smoothing loss, For the i-th grid vertex, Let j be the set of vertices in the neighborhood of vertex i, and j be the set of vertices in the neighborhood. any vertex index in the array, Let be the coordinates of the j-th vertex in the neighborhood; the mesh self-intersection penalty term is used to detect and eliminate the interpenetration relationship between triangular faces that occurs during the optimization process, and its loss is defined as: （15） In the formula, For grid self-intersection penalty loss, and For any two distinct triangular facets, This indicates a pair of facets that intersect or cross each other. A letter of punishment for self-commissioning; Combining all constraints, the overall objective function is obtained. : （16） In the formula, It is the weight of Chamfer Distance. It is the weight of the normal consistency loss. These are the weights of the attitude prior constraints. These are the weights of the prior constraints on the shape. These are the weights of the local Laplace smoothing constraint. It is the weight of the grid self-intersection penalty loss; Iteratively update mesh parameters using the Adam optimizer The update rules are as follows: （17） In the formula, For learning rate, It is the gradient of the loss function with respect to the grid parameters; After iteration, a non-standard human pose mesh M* that fits the shape of the dense human point cloud P is obtained; Step 3.3: Convert the fitted non-standard human pose mesh M* into a standard T-pose, as follows: Extracting pose parameters from a non-standard human pose mesh M* Separate the global pose and local joint pose, and construct the standard pose parameters for the T-pose. The root joint rotation is set to an identity matrix. Both arm joints are rotated to a horizontally extended position, and both leg joints are rotated to a straight and joined position. The rotations of the remaining joints are set to a neutral position. Then, based on the SMPL-X skin weights, a linear hybrid skinning algorithm is used to adjust the standard pose parameters. The formula applied to the fitted shape mesh is as follows: （18） In the formula, Let be the skinning weight of joint j corresponding to vertex i. for The rotation matrix of joint j. Let J be the position of joint j in M*. To determine the standard position of joint j in pose T, the transformed pose mesh is locally smoothed to remove vertex overlap or surface wrinkles that may occur during pose transformation, resulting in the final standardized pose human mesh. and its corresponding model parameters.

8. The method for three-dimensional human point cloud reconstruction and automatic measurement based on multi-view images according to claim 7, characterized in that, Step 4 is as follows: First, based on the key point definition automatic measurement algorithm provided by the SMPL-X official documentation, the standardized pose human mesh generated in step 3 is loaded. The model parameters and their corresponding values ​​are used to calculate the difference between the maximum and minimum values ​​of the grid vertices on the Y-axis of the world coordinate system to obtain the height. Then, key dimensions are measured: the three-dimensional coordinates of 24 joints are calculated based on the model parameters, and then the linear dimensions of shoulder width, left and right arm length, and left and right leg length are measured. For circumference measurement, the cross-section search range is first located by the joints, and the hip circumference cross-section with the largest area is found by sliding the virtual horizontal plane. The chest circumference and waist circumference cross-sections are determined by combining the spinal joints. The vertices of each cross-section are extracted and projected onto the XZ plane. The convex hull algorithm is used to calculate the perimeter of the cross-section to complete the chest circumference, waist circumference, and hip circumference measurements. At the same time, ten key landmark points are dynamically generated according to the definition of clothing measurement, including six feature points on the left and right shoulders, left and right hips, and chest and waist cross-sections. The landmark points are located on the human body surface and satisfy symmetry through preliminary positioning of joints, finding grid intersections by ray projection along the Z-axis, cross-section geometry screening, symmetry mirroring, and linear interpolation. Finally, the key dimension indicators and the three-dimensional coordinates of the landmark points are saved as JSON or TXT structured files to generate two visualization charts. One chart shows the human body grid, measurement cross-sections and joints, and the other chart highlights the ten key landmark points.

Images