Human body three-dimensional modeling method and device, electronic equipment and storage medium

Abstract

The invention discloses a human body three-dimensional modeling method and device, electronic equipment and a storage medium, and relates to the technical field of three-dimensional modeling, and the method comprises the steps: generating an initial sampling point set according to SMPL template vertexes; establishing a dynamic mapping relation between the target surface point set and the initial sampling point set to obtain a mapping matching point set; an error compensation field is constructed to adjust the mapping matching point set, and then a geometric distribution feature map of the human body is obtained; and splicing the geometric distribution characteristic graph and the SMPL vertex graph to obtain a three-dimensional model of the human body. According to the method, the geometric distribution feature map and the SMPL vertex map are spliced, and then the three-dimensional model of the human body is generated according to the spliced feature map obtained by splicing, so that decoupling of the geometric distribution feature map and the SMPL vertex map is realized, the calculation efficiency is improved, and the application cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional modeling, and in particular to a human body three-dimensional modeling method and device, electronic equipment and storage medium. BACKGROUND

[0002] The prior art three-dimensional modeling scheme has problems such as loss of details, poor dynamic adaptability, low efficiency, strong data dependency, and industry application bottlenecks. SUMMARY

[0003] The main purpose of the embodiments of the present application is to at least solve one problem existing in the prior art by providing a human body three-dimensional modeling method, device, electronic equipment and storage medium.

[0004] To achieve the above-mentioned purpose, one aspect of the embodiments of the present application provides a human body three-dimensional modeling method, which comprises the following steps:

[0005] Generating an initial sampling point set according to SMPL template vertices;

[0006] Establishing a dynamic mapping relationship between a target surface point set and the initial sampling point set to obtain a mapping matching point set;

[0007] Constructing an error compensation field to adjust the mapping matching point set, and further obtaining a geometric distribution feature map of the human body;

[0008] Splicing the geometric distribution feature map and the SMPL vertex map to obtain a three-dimensional model of the human body.

[0009] In some embodiments, the step of generating an initial sampling point set according to SMPL template vertices comprises the following steps:

[0010] Performing multi-density Monte Carlo sampling on the SMPL template vertices to further generate the initial sampling point set.

[0011] In some embodiments, the step of establishing a dynamic mapping relationship between a target surface point set and the initial sampling point set to obtain a mapping matching point set comprises the following steps:

[0012] Performing k-d tree nearest neighbor search on each target point in the target surface point set and each vertex in the initial sampling point set to further establish a dynamic mapping relationship and obtain the mapping matching point set;

[0013] The expression of the k-d tree nearest neighbor search is:

[0014]

[0015] wherein x0' represents a matching point in the mapping matching point set, p jrepresents the target point in the target surface point set, x i represents the vertex in the initial sampling point set; V represents the initial sampling point set generated according to the SMPL template vertex.

[0016] In some embodiments, the error compensation field is constructed to adjust the mapped matching point set, so as to obtain a geometric distribution feature map of the human body, including the following steps:

[0017] The error compensation field is constructed to adjust the mapped matching point set, so as to obtain the geometric distribution feature map of the human body.

[0018] The expression of the error compensation field is:

[0019] Δx=β t (x1-x0′);

[0020] Wherein, Δx represents the position error, β t represents a parameter for changing the position of the SMPL template vertex, x1 represents a target vertex, and x0' represents a matching point in the mapped matching point set.

[0021] In some embodiments, the geometric distribution feature map is spliced with an SMPL vertex map to obtain a three-dimensional model of the human body, including the following steps:

[0022] The geometric distribution feature map is upsampled to a target scale by an improved U-Net decoder;

[0023] The upsampled geometric distribution feature map and the SMPL vertex map of the target scale are spliced by applying a normalized displacement field to obtain a spliced feature map.

[0024] The expression of the normalized displacement field is:

[0025]

[0026] Wherein, p out represents the spliced feature map, x0' represents a matching point in the mapped matching point set, F(x0') represents the upsampled geometric distribution feature map, and M(x0') represents an SMPL vertex map, represents feature splicing.

[0027] Spatially balanced sampling is performed on the spliced feature map to obtain the three-dimensional model of the human body.

[0028] In some embodiments, the method further includes the following steps:

[0029] adding Gaussian noise to each vertex in the SMPL template vertices, thereby generating a dynamic candidate point set;

[0030] performing geometric consistency verification on the dynamic candidate point set to obtain screened candidate points;

[0031] training a geometric distribution diffusion model according to the screened candidate points; wherein the geometric distribution diffusion model is used to generate the three-dimensional model of the human body according to a spliced feature map obtained by splicing the geometric distribution feature map and an SMPL vertex map.

[0032] In some embodiments, the performing geometric consistency verification on the dynamic candidate point set to obtain screened candidate points comprises the following steps:

[0033] calculating the nearest neighbor Euclidean distance of each candidate point in the dynamic candidate point set to a target surface point set, and retaining first candidate points with the nearest neighbor Euclidean distance lower than a preset threshold;

[0034] calculating the curvature of each candidate point in the dynamic candidate point set locally through a differential geometry operator, and screening second candidate points with a curvature difference less than a preset radian;

[0035] wherein the screened candidate points include the first candidate points and the second candidate points.

[0036] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes a human three-dimensional modeling device, the device comprises:

[0037] a vertex sampling unit configured to generate an initial sampling point set according to SMPL template vertices;

[0038] a vertex matching unit configured to establish a dynamic mapping relationship between a target surface point set and the initial sampling point set to obtain a mapping matching point set;

[0039] a vertex adjustment unit configured to construct an error compensation field to adjust the mapping matching point set, thereby obtaining a geometric distribution feature map of the human body;

[0040] a model generation unit configured to splice the geometric distribution feature map and an SMPL vertex map to obtain a three-dimensional model of the human body.

[0041] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes an electronic device, the electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0042] To achieve the above object, another aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method described above.

[0043] The embodiment of the present application at least has the following beneficial effects:

[0044] The present application can generate an initial sampling point set according to the SMPL template vertex; establish a dynamic mapping relationship between the target surface point set and the initial sampling point set to obtain a mapping matching point set; construct an error compensation field to adjust the mapping matching point set, and further obtain the geometric distribution feature map of the human body; and splice the geometric distribution feature map and the SMPL vertex map to obtain the three-dimensional model of the human body. The present application first splices the geometric distribution feature map and the SMPL vertex map, and then generates the three-dimensional model of the human body according to the spliced feature map obtained by splicing, realizes the decoupling of the geometric distribution feature map and the SMPL vertex map, and further improves the calculation efficiency and reduces the application cost. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0046] Figure 1 A flowchart of a human three-dimensional modeling method provided by the embodiment of the present application is shown in the figure.

[0047] Figure 2 An example flowchart of a human three-dimensional modeling method provided by the embodiment of the present application is shown in the figure.

[0048] Figure 3 A structural schematic diagram of a human three-dimensional modeling device provided by the embodiment of the present application is shown in the figure.

[0049] Figure 4 A hardware structural schematic diagram of an electronic device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0050] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0051] It can be understood that the terms "first", "second", and the like used in the present application can be used herein to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".

[0052] The terms "at least one", "multiple", "each", "any" and the like used in the present application include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0054] Before the embodiments of the present application are described in detail, first, some related technologies involved in the embodiments of the present application are described as follows:

[0055] Problems existing in the prior art:

[0056] 1. Loss of details: traditional methods (such as NeRF, implicit function) are difficult to capture high-frequency garment details (such as wrinkles), resulting in over-smoothed generated model surfaces.

[0057] 2. Poor dynamic adaptability: point cloud and voxel methods cannot effectively model the dynamic interaction of clothing and human joints, resulting in distorted details when the pose changes.

[0058] 3. Efficiency bottleneck: methods based on rendering optimization (such as NeRF) have high computational overhead and cannot meet the real-time generation requirements.

[0059] 4. Strong data dependency: Existing generative models require multi-view data or complex post-processing, limiting their application in single-view input scenarios.

[0060] 5. Industry application bottleneck: Existing engines (such as Unity / Unreal) have human generation plugins with single character licensing costs as high as 0.5-2, and do not support certain chip architectures. The present application can reduce generation costs by 80% and is compatible with self-controlled hardware platforms.

[0061] Some optional related technologies are introduced as follows:

[0062] gDNA: generates human bodies through implicit functions, but it is difficult to preserve the details of loose clothing.

[0063] E3Gen: based on Gaussian splashing, the generated results have geometric inconsistency.

[0064] Tri-plane + NeRF: limited by resolution, cannot synthesize high-fidelity surfaces.

[0065] Therefore, the embodiments of the present application provide a human three-dimensional modeling method and device, electronic equipment and storage medium. The technical scheme of the present application comprises: generating an initial sampling point set according to the SMPL template vertex; establishing a dynamic mapping relationship between the target surface point set and the initial sampling point set to obtain a mapping matching point set; constructing an error compensation field to adjust the mapping matching point set, and then obtaining the geometric distribution feature map of the human body; and splicing the geometric distribution feature map and the SMPL vertex map to obtain a three-dimensional model of the human body. The present application first splices the geometric distribution feature map and the SMPL vertex map, and then generates a three-dimensional model of the human body according to the spliced feature map, realizes the decoupling of the geometric distribution feature map and the SMPL vertex map, and thus improves the calculation efficiency and reduces the application cost.

[0066] The embodiment of the application provides a human body three-dimensional modeling method, device, electronic equipment and storage medium, relates to the three-dimensional modeling technical field. The human body three-dimensional modeling method, device, electronic equipment and storage medium provided by the embodiment of the application can be applied to a terminal, can also be applied to a server, and can be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal and the like, but is not limited thereto; the server end can be configured as an independent physical server, can also be configured as a server cluster or a distributed system formed by multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN and big data and artificial intelligence platform, and the server can also be a node server in a blockchain network; the software can be an application for implementing a human body three-dimensional modeling method, and the like, but is not limited to the above forms.

[0067] The application can be used in many general or specific computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0068] Reference Figure 1 The embodiment of the application provides a human body three-dimensional modeling method, which can include but is not limited to S100 to S130, and specifically as follows:

[0069] S100: generating an initial sampling point set according to SMPL template vertices;

[0070] S110: establishing a dynamic mapping relationship between a target surface point set and the initial sampling point set to obtain a mapping matching point set;

[0071] S120: constructing an error compensation field to adjust the mapping matching point set, and further obtaining a geometric distribution feature map of the human body;

[0072] S130: Splice the geometric distribution feature map with the SMPL vertex map to obtain a three-dimensional model of the human body.

[0073] Optionally, the generating an initial sampling point set according to the SMPL template vertex includes the following steps:

[0074] The SMPL template vertex is subjected to multi-density Monte Carlo sampling, thereby generating the initial sampling point set.

[0075] Optionally, the establishing a dynamic mapping relationship between the target surface point set and the initial sampling point set to obtain a mapping matching point set includes the following steps:

[0076] The k-d tree nearest neighbor search is performed on each target point in the target surface point set and each vertex in the initial sampling point set, thereby establishing a dynamic mapping relationship to obtain the mapping matching point set.

[0077] The expression of the k-d tree nearest neighbor search is:

[0078]

[0079] wherein x0' represents a matching point in the mapping matching point set, p j represents the target point in the target surface point set, x i represents the vertex in the initial sampling point set; and V represents the initial sampling point set generated according to the SMPL template vertex.

[0080] Optionally, the constructing an error compensation field to adjust the mapping matching point set, thereby obtaining a geometric distribution feature map of the human body includes the following steps:

[0081] The error compensation field is constructed to adjust the mapping matching point set, thereby obtaining the geometric distribution feature map of the human body.

[0082] The expression of the error compensation field is:

[0083] Δx=β t (x1-x0′);

[0084] wherein Δx represents a position error, β t represents a parameter for changing the position of the SMPL template vertex, x1 represents a target vertex, and x0' represents a matching point in the mapping matching point set.

[0085] Optionally, the splicing the geometric distribution feature map with the SMPL vertex map to obtain a three-dimensional model of the human body includes the following steps:

[0086] up-sampling the geometry distribution feature map through an improved U-Net decoder to a target scale;

[0087] applying a normalized displacement field to splice the up-sampled geometry distribution feature map and the SMPL vertex map of the target scale to obtain a spliced feature map;

[0088] An expression of the normalized displacement field is:

[0089]

[0090] wherein p out represents the spliced feature map, x0' represents a matching point in the mapping matching point set, F(x0') represents the up-sampled geometry distribution feature map, M(x0') represents the SMPL vertex map, represents feature splicing;

[0091] performing spatial balanced sampling on the spliced feature map to obtain the three-dimensional model of the human body.

[0092] Optionally, the method further comprises the following steps:

[0093] adding Gaussian noise to each vertex in the SMPL template vertex to further generate a dynamic candidate point set;

[0094] performing geometric consistency verification on the dynamic candidate point set to obtain screened candidate points;

[0095] training a geometry distribution diffusion model according to the screened candidate points; wherein the geometry distribution diffusion model is used to generate the three-dimensional model of the human body according to a spliced feature map obtained by splicing the geometry distribution feature map and the SMPL vertex map.

[0096] Optionally, the performing geometric consistency verification on the dynamic candidate point set to obtain screened candidate points comprises the following steps:

[0097] calculating the nearest neighbor Euclidean distance of each candidate point in the dynamic candidate point set to a target surface point set, and retaining a first candidate point whose nearest neighbor Euclidean distance is lower than a preset threshold;

[0098] calculating the curvature of each candidate point in the dynamic candidate point set through a differential geometry operator, and screening a second candidate point whose curvature difference is less than a preset radian;

[0099] wherein the screened candidate points comprise the first candidate point and the second candidate point.

[0100] Next, the scheme of the embodiments of the present application will be described and explained in detail in combination with specific application examples.

[0101] The embodiment relates to the technical field of computer vision and three-dimensional modeling, and particularly relates to a high-fidelity three-dimensional human body generation method and system based on a geometric distribution diffusion model, which is suitable for a virtual fitting system, dynamic human body modeling in medical simulation training, and real-time role generation in a meta universe scene, and fills a technical blank of existing technology in actual commercial applications.

[0102] With reference to Figure 2 The embodiment proposes a two-stage generation scheme, which realizes high-fidelity and dynamically coupled three-dimensional human body generation by combining geometric distribution modeling and a diffusion model, and solves defects of existing methods in detail retention, posture adaptability and efficiency.

[0103] The core process includes:

[0104] Stage 1 (geometric distribution generation):

[0105] Input data: SMPL template vertex coordinate set

[0106] Processing process:

[0107] Multi-density Monte Carlo sampling (50-500 points / cm 2 Adjustable parameters) is performed on the surface of the SMPL grid to generate an initial sampling point set

[0108] A dynamic mapping relationship between a target surface point set {p1} and SMPL points is established:

[0109] For each target point p j ∈P1, k-d tree nearest neighbor search is performed:

[0110]

[0111] An error compensation field is constructed: Δx = β t (x1-x0′), wherein

[0112] Output result: A geometric distribution feature map F of 128x128x64 is generated and stored in an AWS S3 distributed storage system.

[0113] Stage 2 (high-fidelity synthesis):

[0114] Input data: geometric feature map F + SMPL vertex map

[0115] Processing process:

[0116] A feature map is up-sampled through an improved U-Net decoder (containing 5 layers of cross-scale links);

[0117] Apply normalized displacement field:

[0118]

[0119] where denotes feature stitching, and the MLP contains 3 fully connected layers (dimensions 256→128→64→3);

[0120] Perform spatial balanced sampling:

[0121] Output result: 500,000 point cloud model, format.glb / .usdz.

[0122] The system applying the method of the embodiment includes the following key technical improvements:

[0123] 1. Industrial-grade error compensation unit:

[0124] Dynamic attenuation coefficient β t Design;

[0125] Hardware acceleration interface (CUDA core utilization > 92%);

[0126] Compatible with the quantization operator of Ascend 910B chip (FP16 precision loss <0.1%).

[0127] 2. Multi-scale spatial sampler:

[0128] Support dynamic density adjustment (50-500 points / cm 2 );

[0129] The spatial division adopts an octree structure (depth 8 layers);

[0130] Memory occupation optimization.

[0131] Moreover, the embodiment also introduces a pose decoupling module:

[0132] In the pose decoupling module, an improved neighbor point propagation (NPP) strategy is adopted, and the implementation process is as follows:

[0133] Step 1: Candidate point generation.

[0134] In each training cycle, based on the SMPL template vertex set V, a Gaussian noise is added to each vertex x0′∈V to generate a dynamic candidate point set where σ is the noise intensity parameter, which supports dynamic adjustment to adapt to different clothing material characteristics.

[0135] Step 2: Bidirectional screening mechanism.

[0136] Perform geometric consistency verification on the candidate point set C:

[0137] Distance constraint: compute the nearest neighbor Euclidean distance of candidate point c j to target surface point set {P1}, keep candidate points that satisfy dist(c j , P1) < ε (ε = 0.5mm).

[0138] Curvature constraint: compute the local curvature of candidate point by differential geometry operator, filter out points with curvature difference less than 0.05 radian, avoid surface discontinuity phenomenon.

[0139] The final retained point set is determined by logical expression keep mask = (dist(c j , P1) < ε) ∧ (curv diff < 0.05 rad).

[0140] Step 3: Dynamic update of training set.

[0141] Add the filtered candidate points C[keep mask ] to the training set S, and gradually enhance the modeling ability of the geometric distribution diffusion model for complex garment wrinkles through iterative optimization (T cycles in total). This process achieves efficient data access through distributed memory pool (Redis cluster), with a single cycle throughput of 10 6 points / second.

[0142] The technical effects are shown in Table 1:

[0143]

[0144] Table 1

[0145] Industrial application innovation:

[0146] Dynamic copyright watermark embedding module (compliant with ISO / IEC23001-8);

[0147] GDPR compliant data processing pipeline;

[0148] Support 5G MEC edge computing architecture.

[0149] More specific implementation is described as follows:

[0150] Example 1 (game character generation):

[0151] 1. Input preprocessing:

[0152] Receive SMPL pose parameters from Unity engine

[0153] Generate joint rotation matrix {R i} ∈ SE(3) through SMPL-X parser.

[0154] 2. Geometric distribution generation:

[0155] Call distributed feature encoding cluster (AWS EC2 p3.16xlarge instance).

[0156] Perform multi-scale sampling:

[0157] sampler = OctreeSampler(resolution [64, 128, 256]);

[0158] points = sampler.sample(smpl_mesh, density = 300 pts / cm 2 ).

[0159] Generate feature map F and store to MongoDB sharded cluster.

[0160] 3. High-fidelity synthesis:

[0161] Load Ascend 910B accelerator driver;

[0162] Perform NDF decoding:

[0163] displacement = ndf_decoder(feature_map, pose_matrix);

[0164] final_cloud = base_points + displacement * beta_t.

[0165] Output model verified by Bullet physics engine and imported into Unreal engine.

[0166] Example 2 (medical simulation application):

[0167] 1. Medical data compliant processing:

[0168] Input CT scan slices (DICOM format);

[0169] Process patient features through GDPR de-identification module;

[0170] Generate anonymized SMPL parameters.

[0171] 2. Pathological feature enhancement:

[0172] Embed lesion markers in the UV channel of feature map F:

[0173]

[0174] Visualize the deformation of the organization through the explainability module.

[0175] 3. Real-time interaction:

[0176] Deployed in 5G MEC edge node (delay < 200ms);

[0177] Support HoloLens 2 AR real-time rendering.

[0178] Some optional implementations are as follows:

[0179] In the step of generating candidate points, the nearest SMPL point x0' is matched to the target surface point x1, and Gaussian noise is added to enhance distribution diversity.

[0180] The pose condition includes a single-view normal image, which is fused with the feature map through a cross-attention mechanism.

[0181] This embodiment can be used for real-time simulation of dynamic clothing in a virtual fitting system.

[0182] This embodiment can be used for a virtual fitting service method, which includes the following technical solutions when applied:

[0183] User body data desensitization processing flow (GDPR Article 35 compliance);

[0184] Real-time cloth physics simulation engine (based on Bullet engine improvement);

[0185] Cross-platform model lightweight output module (supports.glb / .usdz format).

[0186] The technical effect data is shown in Table 2:

[0187] Metrics This example Industry standard Model transfer latency ↓ <200 ms 500ms Concurrent throughput ↑ 1000 req / s 200 req / s Hardware power consumption ↓ 85W 150W

[0188] Table 2

[0189] In Table 2, ↓ represents that the smaller the index is, the better, and ↑ represents that the larger the index is, the better.

[0190] Economic benefit data:

[0191] In the virtual clothing fitting scene, a single server saves electricity by 12,500 yuan per year (calculated at 0.15 yuan / kWh).

[0192] Technical coverage: method, system, storage medium (such as game engine plug-in integrating the method).

[0193] Application scenarios: virtual character generation, film digital person creation, virtual fitting, medical simulation (such as dynamic generation of patient models).

[0194] Technical extension coverage:

[0195] Data security transmission: using quantum key distribution (QKD) technology to realize model encryption;

[0196] Hardware acceleration interface: cooperates with NVIDIA Omniverse platform, and CUDA core utilization rate is > 92%.

[0197] Reference Figure 3 The embodiment of the application further provides a human body three-dimensional modeling device, which can realize the human body three-dimensional modeling method.

[0198] The vertex sampling unit is configured to generate an initial sampling point set according to SMPL template vertices.

[0199] The vertex matching unit is configured to establish a dynamic mapping relationship between the target surface point set and the initial sampling point set, and obtain a mapping matching point set.

[0200] The vertex adjustment unit is configured to construct an error compensation field to adjust the mapping matching point set, and further obtain a geometric distribution feature map of the human body.

[0201] The model generation unit is configured to splice the geometric distribution feature map and the SMPL vertex map, and obtain a three-dimensional model of the human body.

[0202] It can be understood that the contents in the above method embodiments are all applicable to the device embodiments, the device embodiments specifically realize the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0203] The embodiment of the application further provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor realizes the method of the embodiment of the application when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer and the like.

[0204] It can be understood that the contents in the above method embodiments are all applicable to the device embodiments, the device embodiments specifically realize the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0205] Please refer to Figure 4 , Figure 4 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises:

[0206] The processor 401 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0207] The memory 402 can be implemented by a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 402 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 402 and are called and executed by the processor 401 to implement the method of the embodiments of the present application.

[0208] The input / output interface 403 is configured to implement information input and output.

[0209] The communication interface 404 is configured to implement the communication interaction between the device and other devices. The communication can be implemented by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0210] The bus 405 is configured to transmit information between the components (for example, the processor 401, the memory 402, the input / output interface 403, and the communication interface 404) of the device.

[0211] The processor 401, the memory 402, the input / output interface 403, and the communication interface 404 are connected to each other by the bus 405 to realize the communication connection between the components in the device.

[0212] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method of the present application.

[0213] It can be understood that the contents in the above method embodiments are all applicable to the storage medium embodiments. The storage medium embodiments specifically implement the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0214] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely from the processor, which can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0215] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0216] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine certain steps or different steps.

[0217] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0218] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0219] The terms "first", "second", "third", "fourth" and the like used in the specification of the present application and the above-described drawings (if any) are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0220] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, and c can be single or multiple.

[0221] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0222] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0223] In addition, each functional unit in each embodiment of the application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0224] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0225] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for three-dimensional modeling of a human body, characterized in that: The method comprises the following steps: Generate an initial sampling point set based on the SMPL template vertices; Establishing a dynamic mapping relationship between the target surface point set and the initial sampling point set to obtain a mapping matching point set; constructing an error compensation field to adjust the mapping matching point set, thereby obtaining a geometric distribution feature map of the human body; The geometric distribution feature map is spliced ​​with the SMPL vertex map to obtain a three-dimensional model of the human body.

2. A three-dimensional human body modeling method according to claim 1, characterized in that: Generating an initial sampling point set according to the SMPL template vertices includes the following steps: Multi-density Monte Carlo sampling is performed on the SMPL template vertices to generate the initial sampling point set.

3. A three-dimensional modeling method of a human body according to claim 1, characterized in that: The step of establishing a dynamic mapping relationship between the target surface point set and the initial sampling point set to obtain a mapping matching point set comprises the following steps: Performing a kd-tree nearest neighbor search on each target point in the target surface point set and each vertex in the initial sampling point set, thereby establishing a dynamic mapping relationship to obtain the mapping matching point set; The expression of the kd tree nearest neighbor search is: Among them, x0′ represents the matching point in the mapping matching point set, p j represents the target point in the target surface point set, x i represents the vertex in the initial sampling point set; V represents the initial sampling point set generated according to the SMPL template vertex.

4. A three-dimensional human body modeling method according to claim 1, characterized in that: The step of constructing an error compensation field to adjust the mapping matching point set, thereby obtaining a geometric distribution feature map of a human body, comprises the following steps: Constructing the error compensation field to adjust the mapping matching point set, thereby obtaining the geometric distribution feature map of the human body; The expression of the error compensation field is: Δx=β t (x1-x0′); Where Δx represents the position error, β t represents the parameters that change the vertex positions of the SMPL template, x1 represents the target vertex, and x0′ represents the matching point in the mapping matching point set.

5. A three-dimensional human body modeling method according to claim 1, characterized in that: The step of splicing the geometric distribution feature map with the SMPL vertex map to obtain a three-dimensional model of a human body comprises the following steps: Upsampling the geometric distribution feature map to a target scale through an improved U-Net decoder; Applying a normalized displacement field to splice the upsampled geometric distribution feature map and the SMPL vertex map of the target scale to obtain a spliced ​​feature map; The expression of the normalized displacement field is: Among them, p out represents the splicing feature map, x0′ represents the matching point in the mapping matching point set, F(x0′) represents the upsampled geometric distribution feature map, M(x0′) represents the SMPL vertex map, Represents feature splicing; Perform spatial balanced sampling on the spliced ​​feature map to obtain the three-dimensional model of the human body.

6. A method for three-dimensional modeling of a human body according to any one of claims 1 to 5, characterized in that: The method further comprises the following steps: Adding Gaussian noise to each vertex in the SMPL template vertex to generate a dynamic candidate point set; Performing geometric consistency verification on the dynamic candidate point set to obtain screened candidate points; A geometric distribution diffusion model is trained based on the screened candidate points; wherein the geometric distribution diffusion model is used to generate the three-dimensional model of the human body based on a spliced ​​feature map obtained by splicing the geometric distribution feature map and the SMPL vertex map.

7. A three-dimensional human body modeling method according to claim 6, characterized in that: The step of performing geometric consistency verification on the dynamic candidate point set to obtain screened candidate points includes the following steps: Calculating the nearest neighbor Euclidean distance between each candidate point in the dynamic candidate point set and the target surface point set, and retaining the first candidate point whose nearest neighbor Euclidean distance is lower than a preset threshold; Calculating the local curvature of each candidate point in the dynamic candidate point set by using a differential geometry operator, and screening out a second candidate point whose curvature difference is less than a preset arc; The filtered candidate points include the first candidate point and the second candidate point.

8. A three-dimensional human body modeling device, characterized in that: The device comprises: Vertex sampling unit, used to generate the initial sampling point set according to the SMPL template vertex; A vertex matching unit, configured to establish a dynamic mapping relationship between a target surface point set and the initial sampling point set to obtain a mapping matching point set; a vertex adjustment unit, configured to construct an error compensation field to adjust the mapping matching point set, thereby obtaining a geometric distribution feature map of the human body; The model generation unit is used to splice the geometric distribution feature map with the SMPL vertex map to obtain a three-dimensional model of the human body.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images