Three-dimensional human body model fitting method based on two-dimensional contour interaction and semantic segmentation

Abstract

This invention discloses a method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation. The method first parses a user-drawn 2D stick figure skeleton using a JPN module and drives an SMSLX model to generate an initial 3D human body. Then, the initial model contour is rendered from multiple fixed perspectives, generating corresponding bounding voxel spaces and point cloud projections. A prior network combining UDF and positional information is introduced to guide the training of the multi-view semantic segmentation network module UPS, achieving accurate segmentation of each part of the contour image. Based on the segmentation results and voxel point cloud projections, surface constraint point clouds for each body part are calculated. Finally, using the constraint point clouds as targets, the SMSLX model parameters are optimized for each part using a CFM module, ultimately generating a 3D human body shape that matches the user's editing intent. This invention allows users to indirectly adjust complex 3D models through intuitive 2D contour editing, significantly improving interaction efficiency and avoiding direct manipulation of complex 3D data.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics, computer vision, and human-computer interaction, and in particular to a method for fitting a three-dimensional human body model based on two-dimensional contour interaction and semantic segmentation. Background Technology

[0002] With the rapid development of virtual reality, digital twins, and the digital entertainment industry, the demand for creating and editing 3D human body models is increasing. However, traditional 3D modeling software is complex to operate, requiring users to have professional spatial geometry knowledge and to directly manipulate 3D vertices, networks, or complex morphological parameters (such as those in the SMPL family). Adjusting parameters is extremely challenging for ordinary users, severely limiting the popularization and creation efficiency of 3D content.

[0003] To reduce the interactive difficulty of 3D modeling, 3D generation and editing methods driven by 2D sketches have become a research hotspot. Early solutions often relied on realistic multi-view photographs or detailed sketches requiring users to have a certain level of artistic skill. These methods are typically based on differentiable rendering technology, constructing a loss function and inversely optimizing the 3D model parameters by comparing the differences between the generated image and the input photograph / sketch. Although these methods achieve automated fitting from 2D to 3D, they have high requirements for the quality and professionalism of the input materials, failing to meet the needs of non-professional users who want to quickly and freely sketch outlines to express their design intentions.

[0004] Against this backdrop, contour-guided differential rendering demonstrates significant advantages. This method allows users to draw only simple 2D contour lines, and the system can automatically drive the fitting of 3D models, greatly simplifying the prerequisite knowledge and operational skills required by users. However, this technical approach faces two core challenges: First, contour maps are essentially binary images, with sparse or even non-existent gradients in pixel space, making it difficult to effectively transfer gradient information in traditional differentiable rendering processes; second, a single binary contour map lacks explicit semantic information, and the model cannot directly learn and understand the semantic composition of different parts (such as the head, arms, and torso) in the contour, thus making it difficult to achieve precise control of different parts.

[0005] After obtaining semantically informational 2D contour constraints, the key lies in how to transform them into geometric constraints on the model surface in 3D space. Surface point cloud guidance becomes an effective strategy for this purpose. Its core idea is to deduce the set of spatial points that the 3D model surface should fit from the 2D contour, using this as a target to drive model fitting. However, how to obtain the surface constraint point cloud based on limited 2D contour and camera information remains a problem to consider. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention aims to provide a method for fitting 3D human body models based on 2D contour interaction and semantic segmentation, solving the problems of complex operation and high professional requirements in existing 3D modeling techniques. This method enables users to quickly generate 3D human body models that meet their intentions through simple and intuitive 2D contour drawing, significantly lowering the technical threshold for 3D content creation.

[0007] This invention is achieved through the following technical solution: A method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation includes the following steps: Step 1: Generate an initial stick figure joint diagram using the initial parametric human body model SMPLX. The user adjusts the joint information of the initial stick figure joint diagram in a two-dimensional plane. The joint change information of the two-dimensional joint is analyzed by the joint point prediction network JPN to determine the user's adjustment intention and drive the parametric human body model SMPLX to fit and generate an initial three-dimensional human body model that conforms to the joint changes. Step 2: Render the initial 3D human body model from multiple fixed perspectives and obtain the outline of the 3D human body; Based on the initial 3D human body model obtained in Step 1, use a fixed perspective camera to generate its corresponding bounding voxel space and the 2D image coordinate projection of the voxel point cloud under each perspective. Step 3: The Unified Prior Guidance Segmentation Module (UPS) performs semantic segmentation on the user contour maps from the front, left, right, and top views to obtain independent binary contour maps of each part under each view. For each target part, two of the most representative contour maps with orthogonal views are selected from its multi-view segmentation results, and its projection constraints under the two orthogonal views are calculated. The generated pair of view constraints will provide boundary and shape guidance information for the subsequent generation of the surface constraint point cloud of the part. Step 4: Based on the constraint information of the two perspectives of different parts obtained in Step 3, process the data in the corresponding voxel space, filter and generate voxel points that satisfy the perspective constraints, so as to jointly constitute the target surface constraint point cloud of that part. Step 5: Based on the target surface constraint point cloud obtained in Step 4, use the constraint fitting module CFM to adjust the initial 3D human body model obtained in Step 1. By optimizing the betas parameters of the parameterized human body model SMPLX model, the target human body model is finally fitted and generated.

[0008] Furthermore, the joint prediction network module JPN mentioned in step one is used to predict the two-dimensional joint information for generating the stick figure image, and the specific process is as follows: The pixel feature information of the stick figure binary image input is concatenated with the normalized coordinates of each pixel relative to the image center on the feature dimension. The specific formula is as follows: ; in X 1 It is the output of stick figure data spliced ​​with relative coordinates in the feature channels; This represents the original pixel data of the input binary stick figure image; Coord x It is the X component of the relative coordinate. Coord y It is the Y component of the relative coordinate. Cat It's a splicing operation; Design a simple encoder-decoder structure. The encoder structure is used to learn the features of the relative positions and shapes of the stick figure's joints, and the decoder structure is used to reconstruct the joint information represented by the features. Finally, a probability map of each pixel being a joint is generated, and the formula is as follows: ; in It is the two-dimensional coordinate information of the joints output by the JPN module of the Joint Prediction Network. , HeatMap Used to generate the most likely pixel position for the current joint. Encoder Constructed from a group of convolutional modules, Decoder Constructed from a group of transposed convolutional modules, These are the image features obtained in the first step.

[0009] Furthermore, the voxel space generation and voxel space projection point generation described in step two include the following two steps: Step 1: Calculate the bounding box of the initial parametric human model SMPLX, and extend it outward by a constant term. Then based on resolution Uniformly generated voxel points, each voxel point representing a spatial region, at a resolution... tend At this time, the voxel points are infinitely dense, approximating a continuous point space. The specific formula for generating the voxel point set is as follows: ; in It is a voxel point set. For three-dimensional indexing, , Let be the boundary value of the bounding box, and its calculation formula is: ; in I unit vector s The bounding box range, c The center of the bounding box; It is a bounding box expansion factor, a positive real constant used to control the extent to which the bounding box of the original 3D human body model expands outward; The step size vector is calculated using the following formula: ; Step 2: Project the voxel point set generated in Step 1 through four orthogonal camera models with fixed viewpoints, and calculate the corresponding two-dimensional screen coordinate point set. The projection process includes the following steps: First, the voxel coordinates are normalized relative to the bounding box center and multiplied by a scaling factor, as shown in the following formula: ; in v Let the coordinates be those of the voxel points. c Center of the enclosure box s For the size of the bounding box, This is the scaling factor; The normalized set of voxels; Secondly, there is the camera coordinate system transformation: for each orthogonal camera with a fixed viewpoint, the camera's rotation matrix is ​​used. Translation vector The normalized voxel points are then transformed to the camera coordinate system using the following formula: ; Finally, the points in the camera's camera space coordinate system are projected onto the two-dimensional screen coordinates. The specific formula is as follows: ; in For screen coordinates, , , which is the camera focal length parameter. , , where are the coordinates of the principal point Let be the coordinates of the point in the camera coordinate system.

[0010] Specifically, in step three, the frontal view is segmented as: head, body, left arm, right arm, left leg, and right leg; the left-side view is segmented as: head, body, and right leg; the right-side view is segmented as: left leg; and the top-side view is segmented as: left arm and right arm.

[0011] Furthermore, the Unified Prior Guidance Segmentation Module (UPS) in step three is a self-designed neural network architecture for generating multi-view contour maps. This module consists of a part prior encoding network and a multi-branch view-specific semantic segmentation network, and obtains segmentation model weights for different views, including but not limited to front, side, and top views, through pre-training. The view-specific semantic segmentation network uses U-Net as its backbone network, decodes by fusing contour, unsigned distance field, and positional encoding features, and receives feature embedding guidance from the part prior encoding network during training. Its training process is as follows: Construct a part prior encoding network, which consists of a set of independent encoders pre-trained for specific human parts, forming a distributed part prior knowledge base; each part encoder specifically learns the morphological features of the corresponding human part, learns the morphological priors of different human parts through supervised training, and outputs a feature embedding vector that integrates global semantics and local details, which is used to guide the training of subsequent semantic segmentation networks. A viewpoint-specific semantic segmentation network is constructed to process front, left, right, and top views respectively. The network is a semantic segmentation model based on the U-Net architecture. The number of network output channels is customized according to different viewpoints. The number of network output channels for front, left, right, and top segmentation networks is set to 6, 3, 2, and 2, respectively, corresponding to the number of parts to be distinguished in each viewpoint. The network achieves accurate part segmentation through a multi-task loss function.

[0012] Furthermore, the specific design of the aforementioned part prior coding network is as follows: (a) Input: For each part, the input consists of four channels of data: a binary contour map of the part, an unsigned distance field UDF map corresponding to the contour, a normalized X-coordinate grid, and a normalized Y-coordinate grid; wherein the normalized X-coordinate is calculated using the following formula: ; (b) Encoder Structure: Each encoder section adopts an encoder-decoder architecture; the encoder section contains five downsampling blocks, each downsampling block consists of two convolutional layers and one max pooling layer, and the kernel size of the convolutional layers is... Step size 1, fill size 1, pooling size of the maximum pooling layer The stride is 2; the convolutional layer is followed by a batch normalization layer and a ReLU activation function; the input image size is... After five downsampling blocks, the feature map space size decreases sequentially to , , , Final output The feature map; the number of channels gradually increases from the initial 32 to 512; the encoder end connects a global average pooling layer and a fully connected layer, outputting a 256-dimensional part feature embedding vector; (c) Decoder Structure: The decoder section is used for supervised learning during the training phase and contains five upsampling blocks; each upsampling block consists of one transposed convolutional layer and two convolutional layers, wherein the kernel size of the transposed convolutional layer is... Step size 2, padding 1; kernel size of the convolutional layer The final output is the reconstruction results of two channels, namely the contour map and the unsigned distance field UDF map; (d) Training strategy: Each region encoder is trained independently using a dataset containing only the contour of that region; the training objective is to minimize the reconstruction loss, including the binary cross-entropy loss of the contour map and the loss of the unsigned distance field UDF map. L1 loss: ; in , These are the weighting coefficients; The binary cross-entropy loss Bce is calculated as follows: ; Where N is the total number of pixel positions in a single feature map, i.e. ; Indicates the pixel position j The true binary labels are 1, which represents that the part belongs to the contour, and 0, which represents that the part does not belong to the contour. This represents the network's prediction at the pixel location. j The probability of belonging to the contour.

[0013] Specifically, the training strategy for the perspective-specific semantic segmentation network is as follows: (a) Network Input: The input consists of four-channel fused features, including: a user-edited complete contour binary image, an unsigned distance field UDF image corresponding to the contour, a normalized X-coordinate grid, and a normalized Y-coordinate grid; the unsigned distance field is calculated using the following formula: ; in For the set of outline pixels, It is during this traversal process that the contour set is formed. The coordinates of the contour points whose distances are currently being calculated are then used; then multimodal feature fusion is performed: the input contour map is... Its corresponding unsigned distance field Feature channels are concatenated, and positional encoding information is introduced to enhance spatial awareness, resulting in a fused feature map. The specific calculation formula is as follows: ; in , For a normalized two-dimensional coordinate network, the following formula is used for calculation: ; (b) U-Net architecture design: The fused features are input into a U-Net-based semantic segmentation network. The encoder extracts multi-level features through multi-layer convolution and downsampling, and the decoder restores spatial resolution through upsampling and skip connections. Finally, the predicted part segmentation probability map is output. and bottleneck layer features Where C is the number of parts to be segmented from this perspective, and D is the semantic depth (number of channels) of the feature. and It refers to the spatial extent of the feature, i.e., the compressed height and width; (c) Multi-task loss optimization: A dual supervision mechanism is adopted during training to segment the loss. Using a combination of Dice loss and binary cross-entropy loss: ; The Dice loss is calculated as follows: ; The binary cross-entropy loss Bce is calculated as follows: ； Where C represents the total number of part categories to be segmented, and N represents the total number of pixel positions in a single feature map, i.e. ; Indicates in category i Below, the true binary label of pixel position j is 1, which means it belongs to this part, and 0 means it does not belong to this part; Indicates the network prediction, in category i Next pixel position j The probability of belonging to the target area; Feature alignment loss This is achieved by calculating the similarity between the bottleneck layer features and the pre-trained feature embeddings: ; in For cosine similarity, This represents the batch size, which is the number of samples input into the network for both forward and backward propagation during training. For the embedded features after projection, Let be the set of effective spatial locations; the overall loss function is: ; in To balance the hyperparameters used to adjust the feature alignment loss The relative weight in the overall optimization objective; Through the above training process, semantic segmentation models from different perspectives are finally obtained, and binary contour maps of each constrained part can be accurately output. In actual process, after inputting the user's multi-view editing map, the contour map of the constrained part will be segmented for different perspectives.

[0014] Furthermore, the generation of the target surface constraint point cloud in step four involves generating the surface target point cloud for each predefined human body part through a dual-view collaborative constraint mechanism. The specific process is as follows: First, select the contour segmentation result from the first-person perspective, and generate an internal region mask for that part by filling the contour. The calculation formula is: ; in This is a contour segmentation map of the region from a first-person perspective; based on this mask, the set of voxel points located within the region in the projected coordinates of this perspective is selected. : ; in voxel point Projected coordinates from a first-person perspective; Secondly, the contour segmentation results from the second perspective are selected, and their unsigned range field is calculated. : ; Based on a preset surface distance threshold From internal point set Further filtering reveals the set of surface points located near the contour from a second perspective. : ; Then, switch the roles of the first-person and second-person perspectives and repeat the above process to obtain the surface point set of reverse filtering. ; Finally, the surface point sets obtained from the two directions are combined to generate the final surface constraint point cloud for that location: ; Through the aforementioned dual-view collaborative constraint mechanism, constraint point clouds distributed around the three-dimensional surface of each part are generated, providing spatial constraints for subsequent model optimization.

[0015] Furthermore, the constraint fitting module described in step five is used to optimize the morphological parameters of the parameterized human model SMPLX using surface constraint point clouds. The specific process is as follows: Based on the predefined semantic segmentation mapping, the vertices of the parameterized human body model SMPLX are divided according to human body parts, and a vertex index set and a face index set corresponding to each part are established to realize the structured representation of the model at the semantic level. Based on the surface constraint point clouds generated in step four, a loss function for partial optimization is constructed and iteratively solved, specifically including: First, calculate the specific loss for each part. This loss is a weighted combination of the chamfer distance and the point-to-surface distance, expressed as: ; in , These are the weighting coefficients. The chamfer distance between the point cloud of this part and the corresponding model vertex. This represents the point-to-surface distance from the point cloud of that region to the corresponding model surface. Then, the losses from each part are weighted and summed to form the overall optimization objective: ; in P For the collection of all parts, These are the weighting coefficients for each part; Finally, the morphological parameters of SMPLX are iteratively updated using the gradient descent algorithm. The total loss function is minimized until convergence or the preset number of iterations is reached. Through the above partial optimization mechanism, the shape parameters of the SMPLX are adjusted so that its three-dimensional shape matches the user's input intention to edit the two-dimensional contour.

[0016] The beneficial effects of this invention are as follows: 1) Joint Prediction Network Module JPN achieves intelligent intent recognition: Unlike traditional semantic recognition and probabilistic prediction methods, the JPN module innovatively adds relative position feature information to the feature layer, enabling the network to more accurately understand and predict the joint spatial relationship of the stick figure, and realize precise mapping from simple two-dimensional adjustment to complex three-dimensional changes; 2) Voxel space technology enables efficient 3D reconstruction: This innovative approach applies traditionally used rendering techniques to 3D point cloud generation. By pre-computing and caching projection information, it maintains real-time interactivity while ensuring the integrity and accuracy of spatial sampling. The dense sampling characteristics of voxel space allow it to approximate continuous space, providing a reliable 3D foundation for subsequent surface point cloud generation. 3) Unified Prior-Guided Segmentation Module (UPS) Achieves Accurate Semantic Segmentation: An innovative two-stage training strategy is employed. First, a pre-trained encoding network learns feature embeddings for different body parts. Then, unsigned distance field (UDF) features are introduced into the semantic segmentation network, providing rich shape information through variations in distance from contour boundaries to inside / outside and asymptotically. Simultaneously, location encoding enhances the network's understanding of the relative spatial relationships between body parts. The guiding role of feature embedding enables the network to clearly define its learning objectives during the downsampling stage and accurately reconstruct each constrained body part during the upsampling stage.

[0017] 4) Dual-view constraint surface point cloud generation: Based on the semantic information of the constraint parts obtained from segmentation, the point cloud in voxel space is filtered through a dual-view collaborative constraint mechanism. First, candidate points are selected based on the internal region of the contour from one viewpoint, and then further refined by the UDF boundary distance from the other viewpoint. Finally, the target point cloud distributed around the 3D surface of each part is obtained, providing accurate spatial constraints for model fitting.

[0018] 5) The Constraint Fitting Module (CFM) achieves accurate model fitting: By segmenting the parametric human model (SMPLX) according to semantic parts and employing a part-specific optimization strategy combining point-to-surface distance and chamfer distance, the model surface is effectively pushed to fit the target point cloud. The point-to-surface distance loss ensures that the model contour accurately matches the user's 2D editing intent, achieving natural and accurate fitting results for adjustments such as making the model fatter or thinner.

[0019] Therefore, through the organic cooperation of various innovative modules, this invention establishes a complete technical solution from two-dimensional interaction to three-dimensional fitting, which has the outstanding advantages of intuitive interaction, accurate fitting, and good real-time performance, effectively reducing the technical threshold for three-dimensional content creation. Attached Figure Description

[0020] Figure 1 This is a flowchart of a three-dimensional human body model fitting method based on two-dimensional contour interaction and semantic segmentation according to the present invention. Figure 2 This diagram illustrates the effect of fitting a 3D model through the joint prediction network module JPN after the user interactively edits the joint diagram of the original parametric human body model SMPLX. Figure 3 The diagram illustrates the overall process and effects of users interactively adjusting 3D human body models based on multi-view contour editing. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0022] like Figure 1As shown, this invention employs a phased fitting strategy to progressively and accurately realize the user's editing intentions for a 3D human body model. The first step aims to fit the user's intention to adjust the relative lengths of the joints. The Joint Prediction Network (JPN) module identifies the user's edited 2D skeleton joint information and intelligently analyzes the user's adjustment intentions (such as limb lengthening or shortening), thereby driving the parametric human body model (SMPLX) to generate an initial 3D human body model that conforms to the changes in joint proportions. This initial 3D human body model is a 3D mesh that has undergone joint proportion adjustments but has not yet had its outline refined; it reflects the user's intention to edit limb lengths and serves as the basis for subsequent outline editing and fitting. The second step, based on the initial 3D human body model obtained in the first step, renders the outline of the initial 3D human body model from four fixed perspectives (front, left, right, and top). This process utilizes the orthogonal projection method in computer graphics to project the 3D model onto a 2D image plane and extract its external outline, generating a binary outline map containing only black and white lines. This allows the user to further edit the outline shape based on this visualization. Users can further edit the outline from each perspective based on this. The system uses the Unified Prior Guidance Segmentation (UPS) module to perform semantic segmentation on the edited contours, extracting the contours of constrained parts from various viewpoints, and then generating corresponding surface constraint point clouds in voxel space. Finally, the Constraint Fitting (CFM) module uses the surface constraint point clouds as the target to perform fine fitting on the initial model obtained in the first step, optimizing the morphological parameters of the parametric human model (SMPLX), and ultimately generating a 3D human model that fully conforms to the user's contour editing intentions. The specific steps are as follows: Step 101: Adjusting the contour information is mainly to control the fat-lean relationship of the parts, and the bone length of the parts needs to be fitted and generated in advance. An initial stick figure joint diagram is generated using the initial parametric human model SMPLX. The user adjusts the joint information of the initial stick figure joint diagram in a 2D plane; the joint prediction network JPN analyzes the 2D joint change information, judges the user's adjustment intention, and drives the parametric human model SMPLX to fit and generate an initial 3D human model that conforms to the joint changes. This process includes the following two steps: Step 1: Concatenate the pixel feature information of the stick figure binary image input with the normalized coordinates of each pixel relative to the image center on the feature dimension. The specific formula is as follows: ; in X 1 It is the output of stick figure data spliced ​​with relative coordinates in the feature channels; This represents the raw pixel data of the input binary skeleton (stick figure) image; Coord x It is the X component of the relative coordinate. Coord yIt is the Y component of the relative coordinate. Cat It's a splicing operation.

[0023] Step 2: Design a simple encoding and decoding structure. The encoding structure is used to learn the features of the relative positions and shapes of the stick figure's joints, and the decoding structure is used to reconstruct the joint information represented by the features. Finally, a probability map of each pixel being a joint is generated, and the formula is as follows: ; in It is the two-dimensional coordinate information of the joints output by the Joint Prediction Network (JPN module). , HeatMap Used to generate the most likely pixel position for the current joint. Encoder Constructed from a group of convolutional modules, Decoder It is constructed from a group of transposed convolutional modules. These are the image features obtained in the first step.

[0024] Step 201: The shape fitting of the model depends on the traction of the surface point cloud. High-density voxel point clouds can effectively generate surface-constrained point clouds. Render the 3D human contour map obtained in Step 101 from multiple fixed viewpoints. Based on the initial 3D human model obtained in Step 101, use a fixed-view camera to generate its corresponding bounding voxel space and the 2D image coordinate projection of the voxel point cloud at each viewpoint. Step 1: Calculate the bounding box of the parametric human model (SMPLX) based on the initial SMPLX model, and extend it outward by a constant term. Then based on resolution Uniformly generated voxel points, each voxel point representing a spatial region, at a resolution... tend At this time, the voxel points are infinitely dense, approximating a continuous point space. The specific formula for generating the voxel point set is as follows: ; in It is a voxel point set. For three-dimensional indexing, , Let be the boundary value of the bounding box, and its calculation formula is: ; in I unit vector s The bounding box range, c The center of the bounding box, It is a bounding box expansion factor, a positive real constant used to control the extent to which the bounding box of the original 3D human body model expands outward; The step size vector is calculated using the following formula: ; Step 2: Project the voxel point set generated in Step 1 through four orthogonal camera models with fixed viewpoints, and calculate the corresponding two-dimensional screen coordinate point set. The projection process includes the following steps: First, the voxel coordinates are normalized relative to the bounding box center and multiplied by a scaling factor, as shown in the following formula: ; in v Let the coordinates be those of the voxel points. c Center of the enclosure box s For the size of the bounding box, Scaling factor It is the set of normalized voxel points.

[0025] Secondly, there is the camera coordinate system transformation: for each orthogonal camera with a fixed viewpoint, the camera's rotation matrix is ​​used. Translation vector The normalized voxel points are then transformed to the camera coordinate system using the following formula: ; Finally, the points in the camera's camera space coordinate system are projected onto the two-dimensional screen coordinates. The specific formula is as follows: ; in For screen coordinates, , , which is the camera focal length parameter. , , where are the coordinates of the principal point Let be the coordinates of the point in the camera coordinate system.

[0026] Step 202: Traditional binary images, with only two color values ​​(255 and 0), hinder network learning and make it difficult to distinguish different line categories. The Unified Prior Guidance Segmentation Module (UPS) utilizes a pre-trained network to perform semantic segmentation on user-adjusted contour images from different viewpoints. From the front viewpoint, it primarily segments the head, body, left arm, right arm, left leg, and right leg; from the left viewpoint, it primarily segments the head, body, and right leg; from the right viewpoint, it primarily segments the left leg; and from the top viewpoint, it primarily segments the left and right arms. The UPS module provides constraints for two viewpoints on different body parts. During training, the UPS module first pre-trains on the reconstruction of the body parts, allowing the network to learn the feature information of different parts. This pre-trained weight is then used to guide the segmentation network training, and unsigned distance field (UDF) features are introduced to help the network learn the sketch features of different body parts effectively. The training process includes the following two steps: Step 1: Learn morphological priors for different human body parts through supervised training, and output a feature embedding vector that integrates global semantics and local details to guide the training of the subsequent semantic segmentation network. The specific network design is as follows: (a) Input: For each part, the input consists of four channels of data: (1) a binary image of the contour of that part; (2) an unsigned distance field (UDF) image of the contour; (3) a normalized X-coordinate grid; and (4) a normalized Y-coordinate grid. The normalized coordinates are calculated using the following formula: ; in, W This represents the width of the input image, which is the total number of pixels in the horizontal direction. H This represents the height of the input image, which is the total number of pixels in the vertical direction. Represents the column index of the current pixel in the image, that is, the integer coordinate of the pixel in the horizontal direction, with a value range of 1 to 2. , y Represents the row index of the current pixel in the image, that is, the integer coordinate of the pixel in the vertical direction, with a value range of 1 to 2. .

[0027] (b) Encoder Structure: Each encoder section employs an encoder-decoder architecture. The encoder section contains five downsampling blocks, each consisting of two convolutional layers (kernel size...). (step size 1, fill size 1) and a max pooling layer (pooling size) The convolutional layer consists of a stride of 2). A batch normalization layer and a ReLU activation function follow the convolutional layer. The input image size is... After five downsampling blocks, the feature map space size decreases sequentially to , , , Final output The feature map is generated. The number of channels is gradually increased from the initial 32 to 512. The encoder ends with a global average pooling layer and a fully connected layer, outputting a 256-dimensional part feature embedding vector.

[0028] (c) Decoder Structure: The decoder section is used for supervised learning during the training phase and contains five upsampling blocks. Each upsampling block consists of a transposed convolutional layer (kernel size...). (stride 2, padding 1) and two convolutional layers (kernel size) The system consists of two channels, which are used to output the reconstruction results (contour map and unsigned distance field map).

[0029] (d) Training Strategy: Each region encoder is trained independently using a dataset containing only the contour of that region. The training objective is to minimize the reconstruction loss, which includes the binary cross-entropy loss of the contour map and the loss of the unsigned distance field (UDF) map. L1 loss: ; in , Here are the weighting coefficients. The binary cross-entropy loss (Bce) is calculated as follows: ; Where N is the total number of pixel positions in a single feature map (i.e., ); Indicates the pixel position j The true binary labels (1 represents that it belongs to the contour of this part, and 0 represents that it does not belong to the contour of this part). This represents the network's prediction at the pixel location. j The probability of belonging to the contour.

[0030] Step 2: Construct a viewpoint-specific semantic segmentation network. The number of output channels is dynamically adjusted for different viewpoints: Frontal view: 6 channels for segmenting head, torso, left arm, right arm, left leg, and right leg; Left side view: 3 channels for segmenting head, torso, and right leg; Right side view: 2 channels for segmenting torso and left leg; Top view: 2 channels for segmenting left arm and right arm. The specific network structure and training strategy are as follows: (a) The network is a semantic segmentation model based on the U-Net architecture, which is customized for different perspectives (front, left, right, top) and achieves accurate part segmentation through a multi-task loss function.

[0031] (b) Network Input: The input consists of four-channel fused features, including: (1) a user-edited complete contour binary image; (2) an unsigned distance field (UDF) image corresponding to the contour; (3) a normalized X-coordinate grid; and (4) a normalized Y-coordinate grid. The unsigned distance field is calculated using the following formula: ; in A set of outline pixels; It is during this traversal process that the contour set is formed. The coordinates of the contour point whose distance is currently being calculated; and It is the spatial extent of the feature (compressed height and width).

[0032] Then, multimodal feature fusion is performed: the input contour map is... Its corresponding unsigned distance field Feature channels are concatenated, and positional encoding information is introduced to enhance spatial awareness, resulting in a fused feature map. The specific calculation formula is as follows: ; in , For a normalized two-dimensional coordinate network, the following formula is used for calculation: ; (c) U-Net architecture design: The fused features are input into a U-Net-based semantic segmentation network. The encoder extracts multi-level features through multi-layer convolution and downsampling, and the decoder restores spatial resolution through upsampling and skip connections. Finally, the predicted part segmentation probability map is output. and bottleneck layer features , where C is the number of parts that need to be segmented from this perspective; (d) Multi-task loss optimization: A dual supervision mechanism is adopted during training to segment the loss. Using a combination of Dice loss and binary cross-entropy loss: ; The Dice loss is calculated as follows: ; The binary cross-entropy loss (Bce) is calculated as follows: ; Where C represents the total number of part categories to be segmented. N represents the total number of pixel locations in a single feature map (i.e., ... ). Indicates in category i Below, the true binary label of pixel position j is 1, which means it belongs to this part, and 0 means it does not belong to this part. Indicates the network prediction, in category i Next pixel position j The probability of belonging to the target area.

[0033] Feature alignment loss This is achieved by calculating the similarity between the bottleneck layer features and the pre-trained feature embeddings: ; in, This refers to the batch size, which is the number of samples input into the network for forward and backward propagation at one time during training. For cosine similarity, For the embedded features after projection, Let be the set of effective spatial locations. The total loss function is: ; in To balance the hyperparameters used to adjust the feature alignment loss The relative weight in the overall optimization objective.

[0034] Through the training process described above, semantic segmentation models from different perspectives can be obtained, accurately outputting binary contour maps of each constrained part. In practice, after inputting the user's multi-view edit image, the model will segment the contour maps of the constrained parts according to different perspectives.

[0035] Step 203: The surface point cloud is used to constrain the model's contour for variation fitting. The voxel space is formed based on the segmentation results of two views for different parts obtained in Step 202. Therefore, it is necessary to process the voxel space according to the constraint information of the two views for different parts, filter and generate voxel points that satisfy the view constraints, so as to jointly constitute the target surface constraint point cloud for that part. This target surface constraint point cloud is a collection of three-dimensional spatial points. Its essence is the spatial constraint representing the desired surface position of the target three-dimensional model after user editing, obtained by back-projecting and filtering two-dimensional contour information in three-dimensional space. It is not the final model mesh, but rather precise spatial position guidance information provided for subsequent optimization processes. Its core function is to serve as a direct fitting target when the CFM module optimizes the morphological parameters of the SMSLX model, thereby driving the initial model deformation so that its surface converges to the shape and contour range defined by the point cloud. Its constraint is equivalent to the depth constraint of different views. The process includes four steps: Step 1: Select the contour segmentation result from the first-person perspective, and generate an internal region mask for that part by filling the inside of the contour. The calculation formula is: ; in This is a contour segmentation image of the region from a first-person perspective. Based on this mask, a set of voxel points located within the region in the projected coordinates of this perspective are selected. : ; in voxel point Projected coordinates from a first-person perspective.

[0036] Step 2: Select the contour segmentation results from the second viewpoint and calculate its unsigned distance field. : ; Based on a preset surface distance threshold From internal point set Further filtering reveals the set of surface points located near the contour from a second perspective. : ; Step 3: Switch between the first-person and second-person perspectives, and repeat steps 1 and 2 to obtain the surface point set obtained through reverse filtering. .

[0037] Step 4: Perform a union operation on the surface point sets obtained from the two directions to generate the final surface constraint point cloud for that location: ; Through the aforementioned dual-view collaborative constraint mechanism, constraint point clouds distributed around the three-dimensional surface of each part are generated, providing spatial constraints for subsequent model optimization.

[0038] Step 204: Based on the surface point cloud obtained in Step 203, the initial model obtained in Step 101 is adjusted using the Constraint Fitting Module (CFM module). By optimizing the Betas parameters of the parametric human body model (SMPLX model), the target human body model is finally fitted and generated. The Constraint Fitting Module (CFM module) mainly uses the distance between points and surfaces to form gradient parameters that guide the model for optimization. This is equivalent to stretching the model's contour to the shape required by this invention. The process includes the following two steps: Step 1: Based on the predefined semantic segmentation mapping, divide the vertices of the parameterized human body model (SMPLX model) according to human body parts, establish the vertex index set and face index set corresponding to each part, and realize the structured representation of the model at the semantic level.

[0039] Step 2: Based on the surface constraint point clouds of each part generated in step 203, construct the loss function for partial optimization and solve iteratively, specifically including: First, calculate the specific loss for each part. This loss is a weighted combination of the chamfer distance and the point-to-surface distance, expressed as: ; in , These are the weighting coefficients. The chamfer distance between the point cloud of this part and the corresponding model vertex. This represents the point-to-surface distance from the point cloud of that region to the corresponding model surface.

[0040] Then, the losses from each part are weighted and summed to form the overall optimization objective: ; in P For the collection of all parts, These are the weighting coefficients for each part; It is a weighted multi-part geometric fitting objective that quantifies the difference between the surface constraint point cloud of each part and the model surface into a specific loss, and combines them into an overall objective through adjustable weight coefficients, thereby guiding the morphological parameters of the SMPLX model to be optimized in a refined and structured manner. It is a key mathematical bridge connecting the user's two-dimensional contour constraints and the three-dimensional model deformation.

[0041] Finally, the morphological parameters of the parameterized human body model (SMPLX model) are iteratively updated using the gradient descent algorithm. The total loss function is minimized until convergence or the preset number of iterations is reached. Through the above partial optimization mechanism, the shape parameters of the parametric human body model (SMPLX model) can be adjusted to match its three-dimensional shape with the user's input of the two-dimensional contour editing intent.

[0042] The technical solution of the present invention will be further explained and illustrated by the following embodiments.

[0043] Example 1: This embodiment, following the technical solution of the present invention, verifies the system's joint recognition and model fitting capabilities by simulating user adjustments to the drawing of different human skeletons. Figure 2 The results of comparative experiments are presented. The original joint diagram is shown as the base model state; subsequent diagrams show six adjusted joint diagrams: increased height, lengthened legs, lengthened arms, shortened height, shortened legs, and shortened arms. These adjusted joint diagrams simulate user interaction input, and the Joint Prediction Network (JPN module) enables automatic joint prediction. The results show that the JPN module accurately predicts the position of each joint in the human skeleton. This indicates that the combination of relative position features introduced into the network with image features effectively enhances the model's understanding of joint spatial distribution, thus achieving robust joint position recognition. The corresponding model fitting results show that even when the user adjusts only one side of the limb, the system can generate a bilaterally coordinated change that conforms to human proportions through intelligent intent analysis. Whether it's overall height adjustment or changes in the length of local limbs (such as legs or arms), the system can accurately understand the user's skeletal editing intent through the prediction results of the JPN module and drive the parametric human model (SMPLX model) to generate the corresponding fitting effect. This embodiment verifies the key role of the Joint Prediction Network (JPN module) in understanding user intent and realizing the transformation from 2D joint adjustment to 3D model change.

[0044] Example 2: In this embodiment, the technical solution of the present invention is strictly followed. By simulating the user's contour editing operation on different parts of the human body, the system verifies its performance in three-dimensional shape fitting. Figure 3The comparative experimental results of this embodiment are presented. The left-hand group of images shows the initial state: the original front, top, left, and right-hand view contour maps, all generated by the base model through fixed camera parameters. The right-hand group shows four typical contour editing cases and their corresponding 3D fitting results: (a) the leg and body contours are widened overall; (b) the body contour is narrowed while the hip contour is widened; (c) the arm, leg, and body contours are narrowed simultaneously; (d) the body, leg, and arm contours are widened across the board. It should be noted that as a binary image, the contour map's gradient information only exists in the boundary pixels. This characteristic makes it difficult for traditional methods to directly obtain effective gradient signals from the contour map to guide model optimization. To solve this problem, this invention innovatively adopts a multi-view collaborative constraint mechanism: the left-hand view provides depth constraints for the front and rear surfaces of the body in the front view, while the front view provides spatial constraints for the left and right surfaces, thereby constructing a complete 3D shape constraint system. To achieve accurate part perception, this invention introduces an enhanced representation based on unsigned distance fields (UDFs). UDF effectively overcomes the limitations of binary images in feature learning by providing continuous boundary distance information, enabling the network to clearly perceive the shape features of different parts. To further enhance semantic understanding, a two-stage training strategy is designed: first, the encoder network is pre-trained to learn and generate feature embeddings for different parts; then, these feature embeddings are used to guide the training of the segmentation network in the downsampling stage, enabling the network to establish the correspondence between shape and part; finally, combined with segmentation labels to supervise the upsampling process, accurate semantic segmentation of multi-view contours is achieved. Finally, the Constraint Fitting Module (CFM) comprehensively utilizes optimization objectives such as point-to-surface distance and chamfer distance to achieve model fitting in three-dimensional space. Experimental results show that, for different contour adjustment schemes, this invention can generate high-quality 3D models that conform to the user's editing intentions, verifying the effectiveness and practicality of the proposed technical solution.

[0045] The above examples are merely illustrative of the present invention and do not constitute a limitation on the scope of protection of the present invention. All designs that are the same as or similar to the present invention are within the scope of protection of the present invention.

Claims

1. A method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation, characterized in that, Includes the following steps: Step 1: Generate an initial stick figure joint diagram using the initial parametric human body model SMPLX. The user adjusts the joint information of the initial stick figure joint diagram in a two-dimensional plane. The joint change information of the two-dimensional joint is analyzed by the joint point prediction network JPN to determine the user's adjustment intention and drive the parametric human body model SMPLX to fit and generate an initial three-dimensional human body model that conforms to the joint changes. Step 2: Render the initial 3D human body model from multiple fixed perspectives and obtain the outline of the 3D human body; Based on the initial 3D human body model obtained in Step 1, use a fixed perspective camera to generate its corresponding bounding voxel space and the 2D image coordinate projection of the voxel point cloud under each perspective. Step 3: The Unified Prior Guidance Segmentation Module (UPS) performs semantic segmentation on the user contour maps from the front, left, right, and top views to obtain independent binary contour maps of each part under each view. For each target part, two of the most representative contour maps with orthogonal views are selected from its multi-view segmentation results, and its projection constraints under the two orthogonal views are calculated. The generated pair of view constraints will provide boundary and shape guidance information for the subsequent generation of the surface constraint point cloud of the part. Step 4: Based on the constraint information of the two perspectives of different parts obtained in Step 3, process the data in the corresponding voxel space, filter and generate voxel points that satisfy the perspective constraints, so as to jointly constitute the target surface constraint point cloud of that part. Step 5: Based on the target surface constraint point cloud obtained in Step 4, use the constraint fitting module CFM to adjust the initial 3D human body model obtained in Step 1. By optimizing the betas parameters of the parameterized human body model SMPLX model, the target human body model is finally fitted and generated.

2. The method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation according to claim 1, characterized in that, The joint prediction network module JPN mentioned in step one is used to predict the two-dimensional joint information for generating the stick figure image. The specific process is as follows: The pixel feature information of the stick figure binary image input is concatenated with the normalized coordinates of each pixel relative to the image center on the feature dimension. The specific formula is as follows: ； in X 1 It is the output of stick figure data spliced ​​with relative coordinates in the feature channels; This represents the original pixel data of the input binary stick figure image; Coord x It is the X component of the relative coordinate. Coord y It is the Y component of the relative coordinate. Cat It's a splicing operation; Design a simple encoder-decoder structure. The encoder structure is used to learn the features of the relative positions and shapes of the stick figure's joints, and the decoder structure is used to reconstruct the joint information represented by the features. Finally, a probability map of each pixel being a joint is generated, and the formula is as follows: ； in It is the two-dimensional coordinate information of the joints output by the JPN module of the Joint Prediction Network. , HeatMap Used to generate the most likely pixel position for the current joint. Encoder Constructed from a group of convolutional modules, Decoder Constructed from a group of transposed convolutional modules, These are the image features obtained in the first step.

3. The method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation according to claim 1, characterized in that, Step two, which describes the generation of voxel space and the generation of voxel space projection points, involves the following two steps: Step 1: Calculate the bounding box of the initial parametric human model SMPLX, and extend it outward by a constant term. Then based on resolution Uniformly generated voxel points, each voxel point representing a spatial region, at a resolution... tend At this time, the voxel points are infinitely dense, approximating a continuous point space. The specific formula for generating the voxel point set is as follows: ； in It is a voxel point set. For three-dimensional indexing, , Let be the boundary value of the bounding box, and its calculation formula is: ； in I unit vector s The bounding box range, c The center of the bounding box; It is a bounding box expansion factor, a positive real constant used to control the extent to which the bounding box of the original 3D human body model expands outward; The step size vector is calculated using the following formula: ； Step 2: Project the voxel point set generated in Step 1 through four orthogonal camera models with fixed viewpoints, and calculate the corresponding two-dimensional screen coordinate point set. The projection process includes the following steps: First, the voxel coordinates are normalized relative to the bounding box center and multiplied by a scaling factor, as shown in the following formula: ； in v Let the coordinates be those of the voxel points. c Center of the enclosure box s For the size of the bounding box, This is the scaling factor; The normalized set of voxels; Secondly, there is the camera coordinate system transformation: for each orthogonal camera with a fixed viewpoint, the camera's rotation matrix is ​​used. Translation vector The normalized voxel points are then transformed to the camera coordinate system using the following formula: ； Finally, the points in the camera's camera space coordinate system are projected onto the two-dimensional screen coordinates. The specific formula is as follows: ； in For screen coordinates, , , which is the camera focal length parameter. , , where are the coordinates of the principal point Let be the coordinates of the point in the camera coordinate system.

4. The method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation according to claim 1, characterized in that, In step three, the frontal view is segmented as: head, body, left arm, right arm, left leg, and right leg; the left-side view is segmented as: head, body, and right leg; the right-side view is segmented as: left leg; and the top-side view is segmented as: left arm and right arm.

5. The method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation according to claim 1, characterized in that, In step three, the Unified Prior Guidance Segmentation Module (UPS) is a self-designed neural network architecture for generating multi-view contour maps. This module consists of a part-prior encoding network and a multi-branch view-specific semantic segmentation network. It obtains segmentation model weights for different viewpoints, including but not limited to front, side, and top views, through pre-training. The view-specific semantic segmentation network uses U-Net as its backbone network, decodes contours, unsigned distance fields, and positional encoding features, and receives feature embedding guidance from the part-prior encoding network during training. Its training process is as follows: Construct a part prior encoding network, which consists of a set of independent encoders pre-trained for specific human parts, forming a distributed part prior knowledge base; each part encoder specifically learns the morphological features of the corresponding human part, learns the morphological priors of different human parts through supervised training, and outputs a feature embedding vector that integrates global semantics and local details, which is used to guide the training of subsequent semantic segmentation networks. A viewpoint-specific semantic segmentation network is constructed to process front, left, right, and top views respectively. The network is a semantic segmentation model based on the U-Net architecture. The number of network output channels is customized according to different viewpoints. The number of network output channels for front, left, right, and top segmentation networks is set to 6, 3, 2, and 2, respectively, corresponding to the number of parts to be distinguished in each viewpoint. The network achieves accurate part segmentation through a multi-task loss function.

6. The method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation according to claim 5, characterized in that, The specific design of the prior coding network for the aforementioned location is as follows: (a) Input: For each part, the input consists of four channels of data: a binary contour map of the part, an unsigned distance field UDF map corresponding to the contour, a normalized X-coordinate grid, and a normalized Y-coordinate grid; wherein the normalized X-coordinate is calculated using the following formula: ； (b) Encoder Structure: Each encoder section adopts an encoder-decoder architecture; the encoder section contains five downsampling blocks, each downsampling block consists of two convolutional layers and one max pooling layer, and the kernel size of the convolutional layers is... Step size 1, fill size 1, pooling size of the maximum pooling layer The stride is 2; the convolutional layer is followed by a batch normalization layer and a ReLU activation function; the input image size is... After five downsampling blocks, the feature map space size decreases sequentially to , , , Final output The feature map; the number of channels gradually increases from the initial 32 to 512; the encoder end connects a global average pooling layer and a fully connected layer, outputting a 256-dimensional part feature embedding vector; (c) Decoder Structure: The decoder section is used for supervised learning during the training phase and contains five upsampling blocks; each upsampling block consists of one transposed convolutional layer and two convolutional layers, wherein the kernel size of the transposed convolutional layer is... Step size 2, padding 1; kernel size of the convolutional layer The final output is the reconstruction results of two channels, namely the contour map and the unsigned distance field UDF map; (d) Training strategy: Each region encoder is trained independently using a dataset containing only the contour of that region; the training objective is to minimize the reconstruction loss, including the binary cross-entropy loss of the contour map and the loss of the unsigned distance field UDF map. L1 loss: ； in , These are the weighting coefficients; The binary cross-entropy loss Bce is calculated as follows: ； Where N is the total number of pixel positions in a single feature map, i.e. ; Indicates the pixel position j The true binary labels are 1, which represents that the part belongs to the contour, and 0, which represents that the part does not belong to the contour. This represents the network's prediction at the pixel location. j The probability of belonging to the contour.

7. The method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation according to claim 5, characterized in that, The training strategy for the perspective-specific semantic segmentation network is as follows: (a) Network Input: The input consists of four-channel fused features, including: a user-edited complete contour binary image, an unsigned distance field UDF image corresponding to the contour, a normalized X-coordinate grid, and a normalized Y-coordinate grid; the unsigned distance field is calculated using the following formula: ； in For the set of outline pixels, It is during this traversal process that the contour set is formed. The coordinates of the contour points whose distances are currently being calculated are then used; then multimodal feature fusion is performed: the input contour map is... Its corresponding unsigned distance field Feature channels are concatenated, and positional encoding information is introduced to enhance spatial awareness, resulting in a fused feature map. The specific calculation formula is as follows: ； in , For a normalized two-dimensional coordinate network, the following formula is used for calculation: ； (b) U-Net architecture design: The fused features are input into a U-Net-based semantic segmentation network. The encoder extracts multi-level features through multi-layer convolution and downsampling, and the decoder restores spatial resolution through upsampling and skip connections. Finally, the predicted part segmentation probability map is output. and bottleneck layer features Where C is the number of parts to be segmented from this perspective, and D is the semantic depth (number of channels) of the feature. and It refers to the spatial extent of the feature, i.e., the compressed height and width; (c) Multi-task loss optimization: A dual supervision mechanism is adopted during training to segment the loss. Using a combination of Dice loss and binary cross-entropy loss: ； The Dice loss is calculated as follows: ； The binary cross-entropy loss Bce is calculated as follows: ； Where C represents the total number of part categories to be segmented, and N represents the total number of pixel positions in a single feature map, i.e. ; Indicates in category i Below, the true binary label of pixel position j is 1, which means it belongs to this part, and 0 means it does not belong to this part; Indicates the network prediction, in category i Next pixel position j The probability of belonging to the target area; Feature alignment loss This is achieved by calculating the similarity between the bottleneck layer features and the pre-trained feature embeddings: ； in, This represents the batch size, which is the number of samples input into the network for both forward and backward propagation during training. For cosine similarity, For the embedded features after projection, Let be the set of effective spatial locations; the overall loss function is: ； in To balance the hyperparameters used to adjust the feature alignment loss The relative weight in the overall optimization objective; Through the above training process, semantic segmentation models from different perspectives are finally obtained, and binary contour maps of each constrained part can be accurately output. In actual process, after inputting the user's multi-view editing map, the contour map of the constrained part will be segmented for different perspectives.

8. The method for fitting a 3D human body model based on 2D contour interaction and semantic segmentation according to claim 1, characterized in that, The generation of the target surface constraint point cloud in step four involves generating the surface target point cloud for each predefined human body part through a dual-view collaborative constraint mechanism. The specific process is as follows: First, select the contour segmentation result from the first-person perspective, and generate an internal region mask for that part by filling the contour. The calculation formula is: ； in This is a contour segmentation map of the region from a first-person perspective; based on this mask, the set of voxel points located within the region in the projected coordinates of this perspective is selected. : ； in voxel point Projected coordinates from a first-person perspective; Secondly, the contour segmentation results from the second perspective are selected, and their unsigned range field is calculated. : ； Based on a preset surface distance threshold From internal point set Further filtering reveals the set of surface points located near the contour from a second perspective. : ； Then, switch the roles of the first-person and second-person perspectives and repeat the above process to obtain the surface point set of reverse filtering. ; Finally, the surface point sets obtained from the two directions are combined to generate the final surface constraint point cloud for that location: ； Through the aforementioned dual-view collaborative constraint mechanism, constraint point clouds distributed around the three-dimensional surface of each part are generated, providing spatial constraints for subsequent model optimization.

9. The method for fitting a three-dimensional human body model based on two-dimensional contour interaction and semantic segmentation according to claim 1, characterized in that, The constraint fitting module described in step five is used to optimize the morphological parameters of the parameterized human model SMPLX using surface constraint point clouds. The specific process is as follows: Based on the predefined semantic segmentation mapping, the vertices of the parameterized human body model SMPLX are divided according to human body parts, and a vertex index set and a face index set corresponding to each part are established to realize the structured representation of the model at the semantic level. Based on the surface constraint point clouds generated in step four, a loss function for partial optimization is constructed and iteratively solved, specifically including: First, calculate the specific loss for each part. This loss is a weighted combination of the chamfer distance and the point-to-surface distance, expressed as: ； in , These are the weighting coefficients. The chamfer distance between the point cloud of this part and the corresponding model vertex. This represents the point-to-surface distance from the point cloud of that region to the corresponding model surface. Then, the losses from each part are weighted and summed to form the overall optimization objective: ； in P For the collection of all parts, These are the weighting coefficients for each part; Finally, the morphological parameters of SMPLX are iteratively updated using the gradient descent algorithm. The total loss function is minimized until convergence or the preset number of iterations is reached. Through the above partial optimization mechanism, the shape parameters of the SMPLX are adjusted so that its three-dimensional shape matches the user's input intention to edit the two-dimensional contour.

Images