Human body modeling method and system based on depth implicit detail representation

Abstract

The invention relates to the technical field of graphics, in particular to a human body modeling method and system based on deep implicit detail characterization, and the method comprises the steps: collecting three-dimensional human body point cloud data through a multi-source heterogeneous sensor fusion system, and generating a high-quality original point cloud data set through space-time alignment and fusion processing; the preprocessing module uses a machine learning algorithm to remove noise points and fill a missing region, and a deep convolutional neural network extracts multi-level features. The parametric modeling module constructs a model template by using a nonlinear deformation basis function, and realizes flexible representation of complex postures and deformation of a human body. The detail enhancement module predicts high-resolution detail features by means of a deep implicit function network, and the sense of reality of the model is improved. And the post-processing module carries out smooth processing, texture mapping, illumination compensation and other operations, and adopts a physics-based rendering algorithm to enhance the visual effect. Therefore, the problem that an existing three-dimensional human body modeling method is low in modeling efficiency is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of graphics, and in particular to a human body modeling method and system based on deep implicit detail representation. Background Art

[0002] With the continuous development of computer graphics and computer vision technologies, the demand for 3D human body modeling is growing in fields such as film and television production, game development, virtual reality, and medical rehabilitation. Traditional 3D human body modeling methods rely primarily on manual modeling or parametric modeling based on simple geometric templates. While these methods can meet basic application needs to a certain extent, they have many limitations.

[0003] On the one hand, traditional manual modeling methods require professional modelers to spend a lot of time and effort on manual sculpting and detail adjustment, resulting in low modeling efficiency and high costs, making it difficult to achieve large-scale human body model generation and real-time updates. On the other hand, parametric modeling methods based on simple geometric templates can quickly generate human body models, but due to the limitations of the templates, they often cannot accurately represent the complex form and detailed features of the human body, such as the fine folds of clothing, the dynamic changes of muscles, and the texture details of the skin. As a result, the generated models lack realism and cannot meet the requirements of high-end applications for high-quality human body models. Summary of the Invention

[0004] The purpose of the present invention is to provide a human body modeling method and system based on deep implicit detail representation, aiming to solve the problem of low modeling efficiency of existing three-dimensional human body modeling methods.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a human body modeling system based on deep implicit detail representation, comprising a data acquisition module, a preprocessing module, a feature extraction module, a parametric modeling module, a detail enhancement module, a post-processing module, and a display interaction module, wherein the data acquisition module, the preprocessing module, the feature extraction module, the parametric modeling module, the detail enhancement module, the post-processing module, and the display interaction module are connected in sequence;

[0006] The data acquisition module is used to collect three-dimensional human body point cloud data using a multi-source heterogeneous sensor fusion system;

[0007] The pre-processing module is used to perform filtering and noise reduction, missing value filling and normal vector estimation on the collected original point cloud data set;

[0008] The feature extraction module is used to automatically extract multi-level features of pre-processed point cloud data using a deep convolutional neural network;

[0009] The parametric modeling module is used to map the extracted feature vectors to the model parameter space through an optimization algorithm based on a preset parametric human body model template to obtain the posture parameters, shape parameters and expression parameters of the human body;

[0010] The detail enhancement module is used to construct a detail enhancement network based on a deep implicit function, taking the parameterized modeling results as input and predicting high-resolution detail features of the human body surface through a multi-layer perceptron network structure;

[0011] The post-processing module is used to perform smoothing, texture mapping, and illumination compensation operations on the generated high-resolution human body model to generate a realistic human body model;

[0012] The display interaction module is used to support human body model rendering and interactive display on multiple display terminals.

[0013] Wherein, the data acquisition module includes a multi-source data acquisition unit, a data synchronization unit and a data fusion unit, and the multi-source data acquisition unit, the data synchronization unit and the data fusion unit are connected in sequence;

[0014] The multi-source data acquisition unit is used to trigger the sensor to collect data;

[0015] The data synchronization unit is used to achieve time synchronization and spatial registration of multi-source data;

[0016] The data fusion unit is used to fuse the synchronized data to generate a complete three-dimensional human body point cloud data set.

[0017] The preprocessing module includes a filtering and denoising unit, a missing value filling unit and a normal vector estimation unit, and the filtering and denoising unit, the missing value filling unit and the normal vector estimation unit are connected in sequence;

[0018] The filtering and denoising unit is used to remove noise points in the original point cloud data;

[0019] The missing value filling unit is used to fill the missing areas in the data;

[0020] The normal vector estimation unit is used to estimate the normal vector information of each point.

[0021] The feature extraction module includes a depth convolution unit, a feature pyramid construction unit and a feature enhancement unit, and the depth convolution unit, the feature pyramid construction unit and the feature enhancement unit are connected in sequence;

[0022] The depth convolution unit is used to extract multi-level features of point cloud data;

[0023] The feature pyramid construction unit is used to fuse features of different scales;

[0024] The feature enhancement unit is used to enhance the fused features and generate feature vector representations.

[0025] Wherein, the parametric modeling module includes a model template storage unit, a feature vector mapping unit and a parameter optimization unit, and the model template storage unit, the feature vector mapping unit and the parameter optimization unit are connected in sequence;

[0026] The model template storage unit is used to store parameterized human body model templates;

[0027] The feature vector mapping unit is used to map the feature vector to the model parameter space;

[0028] The parameter optimization unit is used to obtain various parameters of the human body through an optimization algorithm.

[0029] The detail enhancement module includes a deep implicit function network unit, a multi-scale feature fusion unit and a detail feature prediction unit, and the deep implicit function network unit, the multi-scale feature fusion unit and the detail feature prediction unit are connected in sequence;

[0030] The deep implicit function network unit is used to predict high-resolution detail features of the human body surface;

[0031] The multi-scale feature fusion unit is used to fuse features of different scales;

[0032] The detail feature prediction unit is used to generate a human body model rich in detail features.

[0033] In a second aspect, a human body modeling method based on deep implicit detail representation is used in the human body modeling system based on deep implicit detail representation described in the first aspect, comprising the following steps:

[0034] Use a multi-source heterogeneous sensor fusion system to collect 3D human body point cloud data;

[0035] Perform filtering and noise reduction, missing value filling, and normal vector estimation on the collected original point cloud data set;

[0036] Use deep convolutional neural networks to automatically extract multi-level features from pre-processed point cloud data;

[0037] Based on the preset parametric human body model template, the extracted feature vectors are mapped to the model parameter space through the optimization algorithm to obtain the posture parameters, shape parameters and expression parameters of the human body;

[0038] Construct a detail enhancement network based on deep implicit functions, take the parameterized modeling results as input, and predict high-resolution detail features of the human body surface through a multi-layer perceptron network structure;

[0039] The generated high-resolution human body model is smoothed, texture mapped, and illuminated to generate a realistic human body model.

[0040] The present invention provides a human body modeling system based on deep implicit detail representation, comprising a data acquisition module, a preprocessing module, a feature extraction module, a parametric modeling module, a detail enhancement module, a post-processing module and a display interaction module, wherein the data acquisition module, the preprocessing module, the feature extraction module, the parametric modeling module, the detail enhancement module, the post-processing module and the display interaction module are connected in sequence; the data acquisition module is used to collect three-dimensional human body point cloud data using a multi-source heterogeneous sensor fusion system; the preprocessing module is used to perform filtering and noise reduction, missing value filling and normal vector estimation on the collected original point cloud data set; the feature extraction module is used to automatically extract the human body point cloud data using a deep convolutional neural network. The system extracts multi-level features from preprocessed point cloud data. The parametric modeling module uses an optimization algorithm to map the extracted feature vectors to the model parameter space based on a preset parametric human model template, obtaining the human body's posture, shape, and expression parameters. The detail enhancement module constructs a detail enhancement network based on deep implicit functions. Using the parametric modeling results as input, it predicts high-resolution surface detail features of the human body using a multi-layer perceptron network structure. The post-processing module performs smoothing, texture mapping, and illumination compensation on the generated high-resolution human body model to produce a realistic human body model. The display interaction module supports human body model rendering and interactive display on various display terminals. This invention utilizes a multi-source heterogeneous sensor fusion system to collect three-dimensional human body point cloud data and generates a high-quality raw point cloud dataset through spatiotemporal alignment and fusion. The preprocessing module uses a machine learning algorithm to remove noise points and fill in missing areas, while a deep convolutional neural network extracts multi-level features. The parametric modeling module utilizes nonlinear deformation basis functions to construct the model template, enabling flexible representation of complex human postures and deformations. The detail enhancement module utilizes a deep implicit function network to predict high-resolution detail features, enhancing the model's realism. The post-processing module performs smoothing, texture mapping, and lighting compensation, employing a physically based rendering algorithm to enhance visual effects. The display and interaction module supports multi-terminal display and interaction, providing a convenient and immersive experience. This system is efficient, low-cost, and has good scalability and adaptability, thus addressing the low modeling efficiency of existing 3D human body modeling methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a schematic diagram of a human body modeling system based on deep implicit detail representation provided by the present invention.

[0043] Figure 2 It is a schematic diagram of the data acquisition module.

[0044] Figure 3 It is a schematic diagram of the preprocessing module.

[0045] Figure 4 It is a schematic diagram of the feature extraction module.

[0046] Figure 5 It is a schematic diagram of the parametric modeling module.

[0047] Figure 6 It is a schematic diagram of the detail enhancement module.

[0048] Figure 7 It is a schematic diagram of the post-processing module.

[0049] Figure 8 is a schematic diagram showing the interaction module.

[0050] Figure 9 This is a flow chart of a human body modeling method based on deep implicit detail representation provided by the present invention.

[0051] In the figure: 1-data acquisition module, 2-preprocessing module, 3-feature extraction module, 4-parametric modeling module, 5-detail enhancement module, 6-post-processing module, 7-display interaction module, 11-multi-source data acquisition unit, 12-data synchronization unit, 13-data fusion unit, 21-filtering and denoising unit, 22-missing value filling unit, 23-normal vector estimation unit, 31-depth convolution unit, 32-feature pyramid construction unit, 33-feature enhancement unit, 41-model template storage unit, 42-feature vector mapping unit, 43-parameter optimization unit, 51-depth implicit function network unit, 52-multi-scale feature fusion unit, 53-detail feature prediction unit, 61-smoothing processing unit, 62-texture mapping unit, 63-illumination compensation unit, 71-model rendering unit, 72-terminal adaptation unit, 73-interaction control unit. DETAILED DESCRIPTION

[0052] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0053] See also Figures 1 to 8 In a first aspect, the present invention provides a human body modeling system based on deep implicit detail representation, comprising a data acquisition module 1, a preprocessing module 2, a feature extraction module 3, a parametric modeling module 4, a detail enhancement module 5, a post-processing module 6, and a display interaction module 7, wherein the data acquisition module 1, the preprocessing module 2, the feature extraction module 3, the parametric modeling module 4, the detail enhancement module 5, the post-processing module 6, and the display interaction module 7 are connected in sequence;

[0054] The data acquisition module 1 is used to collect three-dimensional human body point cloud data using a multi-source heterogeneous sensor fusion system;

[0055] The pre-processing module 2 is used to perform filtering, noise reduction, missing value filling and normal vector estimation on the collected original point cloud data set;

[0056] The feature extraction module 3 is used to automatically extract multi-level features of the pre-processed point cloud data using a deep convolutional neural network;

[0057] The parametric modeling module 4 is used to map the extracted feature vectors to the model parameter space through an optimization algorithm based on a preset parametric human body model template to obtain the posture parameters, shape parameters and expression parameters of the human body;

[0058] The detail enhancement module 5 is used to construct a detail enhancement network based on a deep implicit function, taking the parameterized modeling results as input and predicting high-resolution detail features of the human body surface through a multi-layer perceptron network structure;

[0059] The post-processing module 6 is used to perform smoothing, texture mapping, and illumination compensation operations on the generated high-resolution human body model to generate a realistic human body model;

[0060] The display interaction module 7 is used to support human body model rendering and interactive display on multiple display terminals.

[0061] In this embodiment, after the data acquisition module 1 collects three-dimensional human body point cloud data, it transmits it to the pre-processing module 2. The pre-processing module 2 processes the original point cloud data set, including filtering and noise reduction, missing value filling and normal vector estimation, to obtain smooth and complete pre-processed point cloud data, and passes the processed data to the feature extraction module 3. The feature extraction module 3 uses a deep convolutional neural network to automatically extract multi-level features of the pre-processed point cloud data, generate feature vector representations, and then pass these feature vectors to the parametric modeling module 4. Based on a preset parametric human body model template, the parametric modeling module 4 maps the feature vectors to the model parameter space through an optimization algorithm to obtain the posture parameters, shape parameters and expression parameters of the human body, and passes these parameters to the detail enhancement module 5. The detail enhancement module 5 constructs a detail enhancement network based on a deep implicit function, takes the parametric modeling results as input, predicts high-resolution detail features of the human body surface, and generates a human body model rich in detail features, and finally passes the enhanced model to the post-processing module 6. The post-processing module 6 performs smoothing, texture mapping, and lighting compensation on the human body model to generate a realistic human body model, and then sends the final human body model to the display and interaction module 7. The display and interaction module 7 is used to render and display the human body model on multiple display terminals, supporting real-time interactive operations. The present invention utilizes a multi-source heterogeneous sensor fusion system to collect 3D human body point cloud data, generating a high-quality raw point cloud dataset through spatiotemporal alignment and fusion processing. The pre-processing module 2 uses a machine learning algorithm to remove noise points and fill missing areas, and a deep convolutional neural network extracts multi-level features. The parametric modeling module 4 uses nonlinear deformation basis functions to construct a model template, enabling flexible representation of complex human body postures and deformations. The detail enhancement module 5 uses a deep implicit function network to predict high-resolution detail features, enhancing the model's realism. The post-processing module 6 performs smoothing, texture mapping, and lighting compensation, and employs a physically based rendering algorithm to enhance the visual effect. The display and interaction module 7 supports multi-terminal display and interaction, providing a convenient and immersive experience. This system is efficient, low-cost, and has good scalability and adaptability, thus addressing the low modeling efficiency of existing 3D human body modeling methods.

[0062] Furthermore, the data acquisition module 1 includes a multi-source data acquisition unit 11, a data synchronization unit 12 and a data fusion unit 13, and the multi-source data acquisition unit 11, the data synchronization unit 12 and the data fusion unit 13 are connected in sequence;

[0063] The multi-source data acquisition unit 11 is used to trigger the sensor to collect data;

[0064] The data synchronization unit 12 is used to achieve time synchronization and spatial registration of multi-source data;

[0065] The data fusion unit 13 is used to fuse the synchronized data to generate a complete three-dimensional human body point cloud data set.

[0066] In this embodiment, the multi-source data acquisition unit 11 is connected to a variety of sensors (structured light camera, ToF camera, inertial measurement unit, etc.), triggering the sensors to collect data, and obtaining three-dimensional point cloud data of different angles and postures of the human body, as well as human body motion posture information, etc. The data synchronization unit 12 receives the data collected by the multi-source data acquisition unit 11 and performs time synchronization and spatial registration processing. Through methods such as timestamp matching, the data collected by each sensor are aligned in time; using coordinate transformation and other technologies, the data of different sensors are unified into the same spatial coordinate system to ensure the consistency of multi-source data. The data fusion unit 13 performs fusion processing on the synchronized multi-source data. Using data fusion algorithms (weighted average, Bayesian fusion, etc.), the data of different sensors are integrated to generate a complete three-dimensional human point cloud data set. This data set combines the advantages of multiple sensors, has high precision and high resolution, and contains dynamic posture information of the human body.

[0067] Furthermore, the preprocessing module 2 includes a filtering and denoising unit 21, a missing value filling unit 22 and a normal vector estimation unit 23, and the filtering and denoising unit 21, the missing value filling unit 22 and the normal vector estimation unit 23 are connected in sequence;

[0068] The filtering and noise reduction unit 21 is used to remove noise points in the original point cloud data;

[0069] The missing value filling unit 22 is used to fill the missing areas in the data;

[0070] The normal vector estimating unit 23 is used to estimate the normal vector information of each point.

[0071] In this embodiment, the filtering and denoising unit 21 receives the original point cloud data output by the data acquisition module 1, and uses an adaptive filtering algorithm based on machine learning to automatically identify and remove noise points in the data, such as outliers and error points, to improve data quality. The missing value filling unit 22 uses a local geometric feature interpolation algorithm to fill in missing areas in the data. Based on the geometric features and distribution patterns of the surrounding points, the point coordinates and attribute information of the missing area are estimated to make the point cloud data more complete. The normal vector estimation unit 23 estimates the normal vector information of each point in the filled point cloud data. The normal vector reflects the local geometric characteristics of the point cloud surface. The normal vector of each point is obtained by analyzing and calculating the geometric relationship of the points in the point neighborhood.

[0072] Furthermore, the feature extraction module 3 includes a depth convolution unit 31, a feature pyramid construction unit 32 and a feature enhancement unit 33, and the depth convolution unit 31, the feature pyramid construction unit 32 and the feature enhancement unit 33 are connected in sequence;

[0073] The depth convolution unit 31 is used to extract multi-level features of point cloud data;

[0074] The feature pyramid construction unit 32 is used to fuse features of different scales;

[0075] The feature enhancement unit 33 is used to enhance the fused features and generate feature vector representations.

[0076] In this embodiment, the deep convolution unit 31 receives the pre-processed point cloud data and uses a deep convolutional neural network (3D-CNN, etc.) to automatically extract multi-level features of the data. Low-level features mainly include geometric shape information (edges, curvatures, etc.), while high-level features include semantic information (body parts, etc.). The deep convolution network gradually extracts deep features of the data through structures such as convolution layers and pooling layers. The feature pyramid construction unit 32 fuses the multi-level features extracted by the deep convolution unit 31 to construct a feature pyramid. The feature enhancement unit 33 enhances the fused features. A feature enhancement algorithm (attention mechanism, etc.) is used to highlight important features, suppress unimportant features, and generate more representative and discriminative feature vector representations, providing better feature input for subsequent parametric modeling.

[0077] Furthermore, the parametric modeling module 4 includes a model template storage unit 41, a feature vector mapping unit 42 and a parameter optimization unit 43, and the model template storage unit 41, the feature vector mapping unit 42 and the parameter optimization unit 43 are connected in sequence;

[0078] The model template storage unit 41 is used to store parameterized human body model templates;

[0079] The feature vector mapping unit 42 is used to map the feature vector to the model parameter space;

[0080] The parameter optimization unit 43 is used to obtain various parameters of the human body through an optimization algorithm.

[0081] In this embodiment, the model template storage unit 41 stores a preset parametric human body model template, which is constructed using a nonlinear deformation basis function and can flexibly represent various complex postures and deformations of the human body. The basic structure and shape characteristics of the human body are defined in the template, providing a basic framework for parametric modeling. The feature vector mapping unit 42 receives the feature vector output by the feature extraction module 3 and maps it to the model parameter space. By establishing a mapping relationship between the feature vector and the model parameters, the extracted feature information is converted into a parameter description of the human body model. The parameter optimization unit 43: optimizes and adjusts the mapped human body posture parameters, shape parameters, expression parameters, etc. through an optimization algorithm (gradient descent method, etc.). With the goal of minimizing the difference between the model output and the real human body data, the parameters are iteratively optimized to obtain more accurate model parameters that are more in line with the actual human body characteristics.

[0082] Furthermore, the detail enhancement module 5 includes a deep implicit function network unit 51, a multi-scale feature fusion unit 52 and a detail feature prediction unit 53, and the deep implicit function network unit 51, the multi-scale feature fusion unit 52 and the detail feature prediction unit 53 are connected in sequence;

[0083] The deep implicit function network unit 51 is used to predict high-resolution detail features of the human body surface;

[0084] The multi-scale feature fusion unit 52 is used to fuse features of different scales;

[0085] The detail feature prediction unit 53 is used to generate a human body model rich in detail features.

[0086] In this embodiment, the deep implicit function network unit 51 receives the modeling result output by the parameterized modeling module 4, takes it as input, and uses the multi-layer perceptron network structure to predict the high-resolution detail features of the human body surface. The deep implicit function network can generate fine detail information (skin texture, clothing wrinkles, etc.) by learning the distribution law of the detail features of the human body surface. It is the multi-scale feature fusion unit 52 that fuses feature information of different scales. A multi-scale feature fusion strategy (jump connection, etc.) is adopted to integrate features of different levels, making the detail features richer and more complete, and enhancing the model's ability to express details. The detail feature prediction unit 53 generates a human body model rich in high-resolution detail features based on the fused feature information. By predicting and reconstructing the detail features, the human body model is made more realistic and natural in surface details, which improves the realism and visual effect of the model.

[0087] Furthermore, the post-processing module 6 includes a smoothing processing unit 61, a texture mapping unit 62 and an illumination compensation unit 63, and the smoothing processing unit 61, the texture mapping unit 62 and the illumination compensation unit 63 are connected in sequence;

[0088] The smoothing processing unit 61 is used to perform smoothing processing on the human body model;

[0089] The texture mapping unit 62 is used to add texture to the model;

[0090] The illumination compensation unit 63 is used to add illumination compensation and material attributes to the model.

[0091] In this embodiment, the smoothing processing unit 61 receives the high-resolution human body model generated by the detail enhancement module 5 and performs smoothing on it. A smoothing algorithm (Gaussian smoothing, etc.) is used to remove noise and sharp edges on the model surface, making the model surface smoother and more natural. The texture mapping unit 62 adds texture to the smoothed model. Texture samples that match the semantic information of the human body model are searched from the texture database, and the texture information is mapped to the model surface through texture mapping technology, so that the model has richer colors and details. The lighting compensation unit 63 adds lighting compensation and material properties to the texture-mapped model. According to the scene lighting conditions and the material characteristics of the model surface, a physically based rendering algorithm is used to perform lighting compensation and material property settings on the model, so that the model has a more realistic visual effect under different lighting environments.

[0092] Furthermore, the display interaction module 7 includes a model rendering unit 71, a terminal adaptation unit 72 and an interaction control unit 73, and the model rendering unit 71, the terminal adaptation unit 72 and the interaction control unit 73 are connected in sequence;

[0093] The model rendering unit 71 is used to render and display the human body model on various display terminals;

[0094] The terminal adaptation unit 72 is used to ensure the display effect of the model on different terminals;

[0095] The interactive control unit 73 is used to support multiple interactive operations and realize real-time control of the human body model.

[0096] In this embodiment, the model rendering unit 71 receives the realistic human body model generated by the post-processing module 6, and renders and displays it on a variety of display terminals (ordinary displays, virtual reality helmets, augmented reality glasses, etc.). An efficient rendering algorithm is used to convert the model into an image or video signal and output it to the display terminal. The terminal adaptation unit 72 ensures the display effect of the model on different display terminals. According to the display characteristics of the terminal (resolution, refresh rate, etc.), the model is adapted and adjusted so that the model can obtain good display quality on various terminals. The interactive control unit 73 supports a variety of interactive operations (gesture recognition, voice control, etc.) to achieve real-time control of the human body model. By identifying and parsing the interactive instructions input by the user, the rotation, scaling, translation and other operations of the model are controlled to provide users with a more convenient and immersive interactive experience.

[0097] See also Figure 9 In a second aspect, a human body modeling method based on deep implicit detail representation is used in the human body modeling system based on deep implicit detail representation described in the first aspect, comprising the following steps:

[0098] S1 uses a multi-source heterogeneous sensor fusion system to collect 3D human point cloud data;

[0099] Specifically, a multi-source heterogeneous sensor system is built, including different types of sensors such as structured light cameras, Time of Flight cameras, and inertial measurement units. Sensor parameters and acquisition environments are set to ensure proper function and coverage of all body parts and postures. The sensors are triggered to collect data, acquiring 3D point cloud data from different angles and postures of the human body, as well as information about human motion and posture. The collected data is then transmitted to the multi-source data acquisition unit 11 of the data acquisition module 1.

[0100] S2 performs filtering, noise reduction, missing value filling, and normal vector estimation on the collected original point cloud data set;

[0101] Specifically, the original point cloud dataset is input into the filtering and denoising unit 21 of the preprocessing module 2. Using a machine learning-based adaptive filtering algorithm, noise points in the data are automatically identified and removed. The denoised data is then transmitted to the missing value filling unit 22, which uses a local geometric feature interpolation algorithm to fill in missing areas in the data. The filled data is then transmitted to the normal vector estimation unit 23, which estimates the normal vector information for each point.

[0102] S3 uses deep convolutional neural networks to automatically extract multi-level features from pre-processed point cloud data;

[0103] Specifically, the preprocessed point cloud data is input into the deep convolution unit 31 of the feature extraction module 3. A deep convolutional neural network (3D-CNN, etc.) is used to perform convolution operations on the data to extract multi-level features. The extracted features are then transferred to the feature pyramid construction unit 32, which constructs a feature pyramid and fuses features at different scales. The fused features are then transferred to the feature enhancement unit 33, which uses a feature enhancement algorithm (such as an attention mechanism) to enhance the features and generate a feature vector representation.

[0104] Based on the preset parametric human body model template, S4 maps the extracted feature vectors to the model parameter space through an optimization algorithm to obtain the posture parameters, shape parameters and expression parameters of the human body;

[0105] Specifically, the feature vector representation is input into the feature vector mapping unit 42 of the parametric modeling module 4. A preset parametric human body model template is retrieved from the model template storage unit 41. Based on the model template, a mapping relationship between the feature vector and the model parameters is established, and the feature vector is mapped into the model parameter space. The mapped model parameters are transmitted to the parameter optimization unit 43, which optimizes and adjusts the parameters using an optimization algorithm (such as gradient descent) to obtain more accurate human body posture parameters, shape parameters, and expression parameters.

[0106] S5 builds a detail enhancement network based on deep implicit functions, takes the parameterized modeling results as input, and predicts the high-resolution detail features of the human body surface through a multi-layer perceptron network structure;

[0107] Specifically, the parameterized modeling results are input into the deep implicit function network unit 51 of the detail enhancement module 5. A multi-layer perceptron network structure is constructed, and the deep implicit function network is used to learn the distribution patterns of detailed features on the human surface. High-resolution detailed features on the human surface are predicted to generate detailed feature information. This detailed feature information is then transmitted to the multi-scale feature fusion unit 52, which fuses features at different scales. The fused features are then transmitted to the detail feature prediction unit 53, generating a human body model rich in high-resolution detailed features.

[0108] S6 performs smoothing, texture mapping, and lighting compensation on the generated high-resolution human body model to generate a realistic human body model.

[0109] Specifically, the high-resolution human body model is input into the smoothing processing unit 61 of the post-processing module 6. The model surface is smoothed using a smoothing algorithm (such as Gaussian smoothing). The smoothed model is transmitted to the texture mapping unit 62, which searches for matching texture samples from the texture database and performs texture mapping. The texture-mapped model is then transmitted to the illumination compensation unit 63, which uses a physically based rendering algorithm to perform illumination compensation and set material properties based on the scene lighting conditions and the material properties of the model surface. A realistic human body model is generated and transmitted to the display and interaction module 7 for display and interaction.

[0110] What is disclosed above is only a preferred embodiment of the human body modeling method and system based on deep implicit detail representation of the present invention. Of course, this cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that implementing all or part of the processes of the above embodiment and making equivalent changes in accordance with the claims of the present invention still fall within the scope of the invention.

Claims

1. A human body modeling system based on deep implicit detail representation, characterized in that: It includes a data acquisition module, a pre-processing module, a feature extraction module, a parametric modeling module, a detail enhancement module, a post-processing module and a display interaction module, wherein the data acquisition module, the pre-processing module, the feature extraction module, the parametric modeling module, the detail enhancement module, the post-processing module and the display interaction module are connected in sequence; The data acquisition module is used to collect three-dimensional human body point cloud data using a multi-source heterogeneous sensor fusion system; The pre-processing module is used to perform filtering and noise reduction, missing value filling and normal vector estimation on the collected original point cloud data set; The feature extraction module is used to automatically extract multi-level features of pre-processed point cloud data using a deep convolutional neural network; The parametric modeling module is used to map the extracted feature vectors to the model parameter space through an optimization algorithm based on a preset parametric human body model template to obtain the posture parameters, shape parameters and expression parameters of the human body; The detail enhancement module is used to construct a detail enhancement network based on a deep implicit function, taking the parameterized modeling results as input and predicting high-resolution detail features of the human body surface through a multi-layer perceptron network structure; The post-processing module is used to perform smoothing, texture mapping, and illumination compensation operations on the generated high-resolution human body model to generate a realistic human body model; The display interaction module is used to support human body model rendering and interactive display on multiple display terminals.

2. The human body modeling system based on deep implicit detail representation according to claim 1, characterized in that: The data acquisition module includes a multi-source data acquisition unit, a data synchronization unit and a data fusion unit, and the multi-source data acquisition unit, the data synchronization unit and the data fusion unit are connected in sequence; The multi-source data acquisition unit is used to trigger the sensor to collect data; The data synchronization unit is used to achieve time synchronization and spatial registration of multi-source data; The data fusion unit is used to fuse the synchronized data to generate a complete three-dimensional human body point cloud data set.

3. The human body modeling system based on deep implicit detail representation according to claim 1, characterized in that: The preprocessing module includes a filtering and denoising unit, a missing value filling unit and a normal vector estimation unit, wherein the filtering and denoising unit, the missing value filling unit and the normal vector estimation unit are connected in sequence; The filtering and denoising unit is used to remove noise points in the original point cloud data; The missing value filling unit is used to fill the missing areas in the data; The normal vector estimation unit is used to estimate the normal vector information of each point.

4. The human body modeling system based on deep implicit detail representation according to claim 1, characterized in that: The feature extraction module includes a depth convolution unit, a feature pyramid construction unit and a feature enhancement unit, and the depth convolution unit, the feature pyramid construction unit and the feature enhancement unit are connected in sequence; The depth convolution unit is used to extract multi-level features of point cloud data; The feature pyramid construction unit is used to fuse features of different scales; The feature enhancement unit is used to enhance the fused features and generate feature vector representations.

5. The human body modeling system based on deep implicit detail representation according to claim 1, characterized in that: The parametric modeling module includes a model template storage unit, a feature vector mapping unit and a parameter optimization unit, and the model template storage unit, the feature vector mapping unit and the parameter optimization unit are connected in sequence; The model template storage unit is used to store parameterized human body model templates; The feature vector mapping unit is used to map the feature vector to the model parameter space; The parameter optimization unit is used to obtain various parameters of the human body through an optimization algorithm.

6. The human body modeling system based on deep implicit detail representation according to claim 1, characterized in that: The detail enhancement module includes a deep implicit function network unit, a multi-scale feature fusion unit and a detail feature prediction unit, and the deep implicit function network unit, the multi-scale feature fusion unit and the detail feature prediction unit are connected in sequence; The deep implicit function network unit is used to predict high-resolution detail features of the human body surface; The multi-scale feature fusion unit is used to fuse features of different scales; The detail feature prediction unit is used to generate a human body model rich in detail features.

7. A human body modeling method based on deep implicit detail representation, used in the human body modeling system based on deep implicit detail representation according to any one of claims 1 to 6, characterized in that: The following steps are involved: Use a multi-source heterogeneous sensor fusion system to collect 3D human body point cloud data; Perform filtering and noise reduction, missing value filling, and normal vector estimation on the collected original point cloud data set; Use deep convolutional neural networks to automatically extract multi-level features from pre-processed point cloud data; Based on the preset parametric human body model template, the extracted feature vectors are mapped to the model parameter space through the optimization algorithm to obtain the posture parameters, shape parameters and expression parameters of the human body; Construct a detail enhancement network based on deep implicit functions, take the parameterized modeling results as input, and predict high-resolution detail features of the human body surface through a multi-layer perceptron network structure; The generated high-resolution human body model is smoothed, texture mapped, and illuminated to generate a realistic human body model.

Images