Metaverse digital human rendering method, device, equipment and storage medium

Abstract

The present application relates to the field of digital human rendering technology, and discloses a method, device, equipment and storage medium for rendering digital human in the Metaverse. The method comprises: performing three-dimensional data scanning of a target person through a preset high-resolution three-dimensional scanning device to obtain point cloud data; constructing a digital human of the target person in the Metaverse according to the point cloud data to obtain an initial digital human model; configuring texture attributes of the initial digital human to obtain a digital human model to be processed; collecting the action data of the target person, and performing model rendering optimization on the digital human to be processed according to the action data to obtain a target digital human model. The present application improves the authenticity of the rendering of digital human in the Metaverse.

Description

Technical Field

[0001] The present application relates to the field of digital human rendering, and in particular to a method, device, equipment and storage medium for rendering a metaverse digital human. Background Art

[0002] In the field of digital human rendering, existing technologies mainly rely on traditional 3D modeling, texture mapping, and simple lighting processing technologies. These technologies can create basic 3D character models, add textures and materials manually, and realize basic action animations. In addition, traditional methods also apply simple ambient lighting simulations to enhance the realism of the scene. This series of technologies provides basic support for fields such as digital entertainment, virtual reality, and augmented reality, allowing digital characters to be used in various virtual environments.

[0003] However, existing technologies have many deficiencies in rendering highly complex digital humans. Traditional 3D modeling and texture mapping methods often fail to achieve highly realistic effects when processing human details (such as skin texture, facial expressions, etc.), especially in complex lighting and dynamic environments. Secondly, the naturalness and smoothness of action animations are difficult to optimize through manual production, which limits the expressiveness and interactivity of digital characters. Simple lighting processing technology cannot accurately simulate the complex interactions of light in the real world, such as reflection, refraction, and scattering, resulting in a lack of depth and layering in the rendered scenes. Summary of the invention

[0004] The present application provides a method, apparatus, device and storage medium for rendering a digital human in a metaverse, which are used to improve the authenticity of the rendering of a digital human in a metaverse.

[0005] In the first aspect, the present application provides a method for rendering a digital human in a metaverse, and the method for rendering a digital human in the metaverse includes: performing a three-dimensional data scan of a target person through a preset high-resolution three-dimensional scanning device to obtain point cloud data; constructing a digital human of the target person in the metaverse according to the point cloud data to obtain an initial digital human model; configuring texture attributes of the initial digital human to obtain a digital human model to be processed; collecting action data of the target person, and performing model rendering optimization on the digital human to be processed according to the action data to obtain a target digital human model.

[0006] In combination with the first aspect, in a first implementation method of the first aspect of the present application, the three-dimensional data scanning of the target person by a preset high-resolution three-dimensional scanning device to obtain point cloud data includes: performing three-dimensional data scanning on the target person by the high-resolution three-dimensional scanning device to obtain initial point cloud data; down-sampling the initial point cloud data to obtain sampled point cloud data; constructing a graph structure according to the sampled point cloud data, and extracting structural data of the graph structure; inputting the structural data into a preset graph neural network to identify noise points to obtain noise point data; denoising the sampled point cloud data based on the noise point data to obtain denoised point cloud data; performing data alignment processing on the denoised point cloud data to obtain the point cloud data.

[0007] In combination with the first aspect, in a second implementation method of the first aspect of the present application, the digital human model of the target person is constructed in the metaverse according to the point cloud data to obtain an initial digital human model, including: splitting the point cloud data into point cloud distribution areas to obtain multiple point cloud distribution areas; performing point cloud distribution density analysis on each of the point cloud distribution areas respectively to obtain the point cloud distribution density of each point cloud distribution area; inputting the point cloud distribution density of each point cloud distribution area into a preset adaptive grid reconstruction algorithm for grid attribute matching to obtain grid attribute data; based on the grid attribute data, a digital human model of the target person is constructed in the metaverse through a preset Poisson surface reconstruction algorithm to obtain a first digital human model; filling holes and defects in the first digital human model to obtain a second digital human model; performing detail enhancement processing on the second digital human model to obtain a third digital human model; and performing multi-perspective fusion optimization on the third digital human model to obtain the initial digital human model.

[0008] In combination with the first aspect, in a third implementation of the first aspect of the present application, the configuring texture attributes of the initial digital human to obtain the digital human model to be processed includes: performing texture image acquisition on the initial digital human to obtain model texture image data; performing high-frequency detail extraction on the model texture image data to obtain high-frequency detail data; performing low-frequency detail extraction on the model texture image data to obtain low-frequency detail data; performing geometric feature analysis on the initial digital human to obtain a geometric feature set of the initial digital human; performing model curvature analysis on the initial digital human based on the high-frequency detail data and the low-frequency detail data to obtain model curvature data; inputting the geometric feature set and the model curvature data into a preset texture mapping algorithm for texture attribute mapping to obtain target texture attributes; and configuring texture attributes of the initial digital human based on the target texture attributes to obtain the digital human model to be processed.

[0009] In combination with the first aspect, in a fourth implementation method of the first aspect of the present application, the action data of the target person is collected, and model rendering optimization of the digital human to be processed is performed according to the action data to obtain a target digital human model, including: collecting the action data of the target person, and performing time series data matching on the action data to obtain a time series data set; extracting key actions from the action data based on the time series data set to obtain key action data; scanning the key action data for action details to obtain an action detail data set; performing detail coherence analysis on the action detail data set to obtain a detail coherence evaluation value; performing detail optimization on the action detail data set based on the detail coherence evaluation value to obtain an optimized detail data set; performing model rendering optimization on the digital human to be processed based on the optimized detail data set to obtain a target digital human model.

[0010] In combination with the first aspect, in a fifth implementation method of the first aspect of the present application, the model rendering optimization of the digital human to be processed based on the optimized detail data set to obtain a target digital human model includes: constructing a lighting model for the digital human to be processed to obtain a target lighting model; performing lighting missing area analysis on the target lighting model through a ray tracing algorithm to obtain a lighting missing area; performing lighting enhancement on the lighting missing area to obtain an enhanced digital human model; performing shadow edge detection on the enhanced digital human model to obtain a shadow outline; performing shadow lighting parameter matching on the shadow outline according to the target lighting model to obtain a shadow lighting parameter set; performing parameter configuration on the enhanced digital human model based on the shadow lighting parameter set to obtain a configured digital human model; performing model rendering optimization on the configured digital human model based on the optimized detail data set to obtain the target digital human model.

[0011] In combination with the first aspect, in a sixth implementation method of the first aspect of the present application, the model rendering optimization of the configured digital human model based on the optimized detail data set to obtain the target digital human model includes: performing action type analysis on the optimized detail data set to obtain an action type set; based on the action type set, data grouping the optimized detail data set to obtain multiple groups of action detail data; performing action parameter analysis on each group of the action detail data to obtain an action parameter set for each group of the action detail data; performing model rendering optimization on the configured digital human model based on the action parameter data of each group of the action detail data to obtain the target digital human model.

[0012] In the second aspect, the present application provides a rendering device for a metaverse digital human, and the rendering device for a metaverse digital human includes: a scanning module, which is used to perform three-dimensional data scanning on a target person through a preset high-resolution three-dimensional scanning device to obtain point cloud data; a construction module, which is used to construct a digital human of the target person in the metaverse according to the point cloud data to obtain an initial digital human model; a configuration module, which is used to configure texture attributes of the initial digital human to obtain a digital human model to be processed; an optimization module, which is used to collect action data of the target person, and perform model rendering optimization on the digital human to be processed according to the action data to obtain a target digital human model.

[0013] The third aspect of the present application provides a rendering device for a metaverse digital human, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the rendering device for the metaverse digital human executes the above-mentioned rendering method for the metaverse digital human.

[0014] A fourth aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned method for rendering a metaverse digital human.

[0015] In the technical solution provided by the present application, by using a preset high-resolution three-dimensional scanning device to scan the three-dimensional data of the target person, the subtle geometric features of the person can be captured, ensuring the high authenticity and accuracy of the digital human model. This provides a solid foundation for building a high-quality metaverse digital human. By downsampling the initial point cloud data, building a graph structure, extracting structural data, identifying and denoising noise points, and aligning data, the quality of the point cloud data is effectively optimized. This not only improves the efficiency of data processing, but also ensures the clarity and accuracy of the geometric details of the final digital human model. Adopting an adaptive mesh reconstruction algorithm and a Poisson surface reconstruction algorithm, combined with detail enhancement processing and multi-view fusion optimization, the mesh can be automatically adjusted according to the distribution of point cloud data, void defects can be filled, and model details can be enhanced. This series of optimization measures greatly improves the visual authenticity and aesthetics of the digital human model. Through the acquisition of texture images, detail extraction, geometric feature analysis and texture mapping algorithms, the texture properties of the digital human model can be accurately configured. In particular, the combination of high-frequency and low-frequency detail data, as well as model curvature analysis, ensures the natural fit of the texture and the consistency of the visual effect. The target person's motion data is collected and analyzed. Through timing matching, key motion extraction, motion detail scanning and coherence analysis, the naturalness and smoothness of the motion details are optimized. This ensures that the digital human's motion performance in the metaverse is both realistic and natural. The target lighting model is constructed and the lighting effect is optimized through ray tracing technology. Combined with shadow contour detection and lighting parameter matching, the three-dimensional sense and presence of the digital human model are further enhanced. Through model rendering optimization, it is ensured that the digital human model shows highly realistic and attractive visual effects under various lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0017] Figure 1 A schematic diagram of an embodiment of a method for rendering a digital human in a metaverse in an embodiment of the present application;

[0018] Figure 2 This is a schematic diagram of an embodiment of a rendering device for a metaverse digital human in an embodiment of the present application. DETAILED DESCRIPTION

[0019] Embodiments of the present application provide a method, device, equipment and storage medium for rendering a digital human in a metaverse. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the method for rendering a digital human in the Metaverse in the embodiment of the present application includes:

[0021] Step S101, performing three-dimensional data scanning on a target person by using a preset high-resolution three-dimensional scanning device to obtain point cloud data;

[0022] Specifically, a high-resolution three-dimensional scanning device is used to perform a three-dimensional scan on the target person, and accurate three-dimensional information of the appearance and form of the target person is obtained to generate initial point cloud data. Since the directly generated point cloud data is often very large and may contain millions of points, in order to improve the efficiency of subsequent processing and reduce the consumption of computing resources, the initial point cloud data is then downsampled to reduce the amount of data while retaining enough information to accurately represent the three-dimensional form of the target person. A graph structure is constructed based on the sampled point cloud data, and structural data of the graph structure is extracted from it. By analyzing the spatial relationship between each point in the point cloud data, the subsequent algorithms are helped to better understand and process the data. After obtaining the graph structure, its structural data is input into a preset graph neural network, and through its recognition of noise points, the noise point data can be effectively identified from the sampled point cloud data. Noise points refer to error points that do not belong to the main body of the target person and may be introduced by the scanning environment or other factors. Based on the noise point data, the sampled point cloud data is denoised to remove unnecessary noise points, and denoised point cloud data that is purer and accurately represents the three-dimensional form of the target person is obtained. In order to ensure that the point cloud data can accurately correspond to the real three-dimensional space of the target person, data alignment processing is performed. Data alignment processing adjusts the positions of the points in the point cloud data to ensure that they correctly match the three-dimensional morphological features of the target person, and finally obtains the point cloud data.

[0023] Step S102: construct a digital human of the target person in the metaverse according to the point cloud data to obtain an initial digital human model;

[0024] It should be noted that the point cloud data is segmented into regions. By analyzing the entire point cloud data set, it is divided into multiple point cloud distribution regions, each of which contains a portion of point cloud data. The point cloud distribution density is analyzed for each point cloud distribution region, and the point cloud distribution density information is obtained by calculating the density of the point cloud in each region, guiding the reconstruction algorithm to adopt different processing strategies in different regions to achieve better reconstruction results. The point cloud distribution density is input into the preset adaptive grid reconstruction algorithm for grid attribute matching. The algorithm uses different grid densities in different regions for reconstruction according to the different point cloud distribution densities to generate grid attribute data. Based on the grid attribute data, the preset Poisson surface reconstruction algorithm is used to construct a digital human model of the target person in the metaverse. The algorithm can effectively use the point cloud data and grid attribute data to generate a continuous and smooth surface, thereby obtaining the first digital human model. The first digital human model is filled with holes and defects. There may still be holes or defects in the model due to incomplete data or limitations of the reconstruction algorithm. The holes and defects are filled by a special algorithm to obtain a more complete second digital human model. In order to further improve the quality of the model, the second digital human model is enhanced in detail, and the realism and visual effect of the digital human model are enhanced by adding surface details or adjusting surface textures, so as to obtain the third digital human model. The third digital human model is optimized by multi-view fusion. By analyzing the performance of the model under different perspectives, the model data is adjusted and optimized to make its performance in the metaverse more natural and realistic, and finally the initial digital human model is obtained.

[0025] Step S103, configuring texture attributes of the initial digital human to obtain a digital human model to be processed;

[0026] Specifically, texture images are collected for the initial digital human, and texture images related to the target person's skin color, clothing, etc. are captured or selected to obtain model texture image data. These data need to contain not only color information, but also sufficient detail information to better simulate the physical properties and visual effects in the real world. By analyzing the model texture image data, high-frequency and low-frequency detail data are extracted respectively. High-frequency detail extraction focuses on capturing tiny details in the texture, such as fine lines on the skin and subtle textures of fabrics, which help enhance the realism of the digital human model. At the same time, low-frequency detail extraction focuses on larger regional changes and smooth transitions in the texture, such as gradual changes in skin color and large-area color changes in clothing, which help form the basic visual impression of the digital human model. Geometric feature analysis is performed to obtain the geometric feature set of the initial digital human by analyzing the shape, volume and surface characteristics of the digital human model. Geometric features determine how the texture is adapted and adjusted according to the specific shape and curvature of the model. Based on the high-frequency detail data and low-frequency detail data, the initial digital human is subjected to model curvature analysis to obtain model curvature data. This analysis helps understand the curvature and complexity of the model surface, ensuring that the texture mapping can accurately fit the model surface, especially on curved or concave parts of the model. The geometric feature set and model curvature data are input into the preset texture mapping algorithm for texture attribute mapping. The algorithm maps the texture image to the surface of the digital human model to ensure the correct position, orientation and size of the texture. Through mapping, the target texture properties are obtained, which include accurate placement of textures, seamless fusion between textures, and realistic presentation of texture details. Based on the target texture properties, the texture properties of the initial digital human are configured, and the texture details are adjusted and optimized to ensure that the generated digital human model to be processed is visually rich and realistic.

[0027] Step S104: collect the action data of the target person, and perform model rendering optimization on the digital human to be processed according to the action data to obtain the target digital human model.

[0028] It should be noted that the motion data of the target person is collected. A high-precision motion capture system is used to record every movement and expression of the target person. The motion data is matched with time series data, and the action is associated with the time point when it occurs to generate a time series data set. Based on the time series data set, the motion data is analyzed, and the key action data is generated by identifying and extracting the key actions in the action sequence. The key action data is scanned for action details to obtain an action detail data set, which contains the subtle changes and features of each key action. The action detail data set is analyzed for detail coherence, and the coherence between the details of each action is evaluated to ensure that the action transition is natural and smooth without unnatural jitter or mutation, and the detail coherence evaluation value is obtained, which is a quantitative evaluation of the coherence of the action details. Based on the detail coherence evaluation value, the action detail data set is optimized in detail, and those parts of the action data that lack coherence are adjusted and improved to make the entire action sequence smoother and more natural. After optimization, an optimized detail data set is obtained. Based on the optimized detail data set, the model rendering of the digital human to be processed is optimized, the optimized action details are applied to the digital human model, and the actions and expressions of the target person are reproduced through advanced rendering technology, finally obtaining the target digital human model.

[0029] Among them, a lighting model is constructed for the digital human to be processed to simulate the effect of light on objects in the real world, ensuring that the digital human model can show realistic visual effects under different lighting conditions. Through the lighting model, a target lighting model is generated, which takes into account factors such as the position, intensity, and color of the light source. The target lighting model is analyzed using a ray tracing algorithm, especially for the analysis of the lighting-deficient areas. The ray tracing algorithm calculates the propagation, reflection, and refraction of light by simulating the interaction between light and the surface of the object, thereby effectively identifying which areas are not fully illuminated under the current lighting model. Special lighting enhancement processing is performed on the lighting-deficient areas, and the lighting effects of these areas are improved by adjusting the lighting conditions or adding virtual light sources, so that the lighting distribution of the entire digital human model is more uniform and natural, and an enhanced digital human model is obtained. Shadow edge detection is performed to understand and simulate the effect of shadows on the digital human model, and the relationship between the surface and the lighting model of the enhanced digital human model is analyzed to accurately determine the outline of the shadow. The shadow lighting parameters of the shadow outline are matched according to the target lighting model, and the appropriate shadow lighting parameter set is determined by analyzing the specific conditions for the shadow generation, such as the position and intensity of the light source. This parameter set is used to adjust and optimize the performance of the shadow in the model to make it more realistic and natural. The enhanced digital human model is parameterized based on the shadow lighting parameter set, and the model's lighting, shadow and other visual effects are adjusted to ensure that the model can show the best visual effects under any lighting conditions, and the configured digital human model is obtained. On this basis, the optimized detail data set is used to perform the final model rendering optimization on the configured digital human model, including further refinement of the model details and the final adjustment of the lighting and shadow effects to ensure the perfect visual presentation of the target digital human model.

[0030] Among them, the action type analysis is performed on the optimized detail data set, and different action types, such as walking, jumping or other specific action performances, are identified from the optimized detail data set to obtain a set that comprehensively covers various action types. This provides a basic classification standard for subsequent data processing and model rendering, ensuring that optimization measures for different types of actions can be accurately applied. Based on the action type set, the optimized detail data set is grouped, and the data is classified according to the action type to which it belongs, to obtain multiple groups of detail data for specific actions. Grouping helps to process data systematically, and can also adopt the most appropriate rendering and optimization strategies for different action types, thereby improving the efficiency and effectiveness of the entire optimization process. Action parameter analysis is performed on each group of action detail data to understand the characteristics and requirements of each action, and the action parameter set of each group of action detail data is obtained through analysis. These action parameters include key factors such as the speed, amplitude, and duration of the action, which are essential for reproducing accurate and natural action performance. Through action parameter analysis, it is ensured that the rendering of different actions can be as close to the real performance as possible, enhancing the realism and dynamic expression of the digital human model. Based on the action parameters of each group of action detail data, the model rendering optimization of the configured digital human model is performed. The action parameters obtained by analysis are applied to the specific rendering process of configuring the digital human model to ensure that each action can be presented in the most realistic form on the model to obtain the target digital human model.

[0031] It is understandable that the execution subject of the present application can be a rendering device of the Metaverse Digital Human, or a terminal or a server, which is not limited here. The present application embodiment is described by taking the server as the execution subject as an example.

[0032] In the embodiment of the present application, by using a preset high-resolution three-dimensional scanning device to scan the three-dimensional data of the target person, the subtle geometric features of the person can be captured, ensuring the high authenticity and accuracy of the digital human model. This provides a solid foundation for building a high-quality metaverse digital human. By downsampling the initial point cloud data, building a graph structure, extracting structural data, identifying and denoising noise points, and aligning data, the quality of the point cloud data is effectively optimized. This not only improves the efficiency of data processing, but also ensures the clarity and accuracy of the geometric details of the final digital human model. Adopting an adaptive mesh reconstruction algorithm and a Poisson surface reconstruction algorithm, combined with detail enhancement processing and multi-view fusion optimization, the mesh can be automatically adjusted according to the distribution of point cloud data, void defects can be filled, and model details can be enhanced. This series of optimization measures greatly improves the visual authenticity and aesthetics of the digital human model. Through the acquisition of texture images, detail extraction, geometric feature analysis and texture mapping algorithms, the texture properties of the digital human model can be accurately configured. In particular, the combination of high-frequency and low-frequency detail data, as well as model curvature analysis, ensures the natural fit of the texture and the consistency of the visual effect. The target person's motion data is collected and analyzed. Through timing matching, key motion extraction, motion detail scanning and coherence analysis, the naturalness and smoothness of the motion details are optimized. This ensures that the digital human's motion performance in the metaverse is both realistic and natural. The target lighting model is constructed and the lighting effect is optimized through ray tracing technology. Combined with shadow contour detection and lighting parameter matching, the three-dimensional sense and presence of the digital human model are further enhanced. Through model rendering optimization, it is ensured that the digital human model shows highly realistic and attractive visual effects under various lighting conditions.

[0033] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0034] (1) Scanning the target person in three dimensions using a high-resolution three-dimensional scanning device to obtain initial point cloud data;

[0035] (2) down-sampling the initial point cloud data to obtain sampled point cloud data;

[0036] (3) construct a graph structure based on the sampled point cloud data and extract the structural data of the graph structure;

[0037] (4) Inputting the structural data into a preset graph neural network to identify noise points and obtain noise point data;

[0038] (5) De-noising the sampled point cloud data based on the noise point data to obtain de-noised point cloud data;

[0039] (6) Perform data alignment on the denoised point cloud data to obtain point cloud data.

[0040] Specifically, a high-resolution three-dimensional scanning device is used to perform a three-dimensional data scan on the target person. The person in the real world is converted into initial point cloud data in digital form. The high-resolution three-dimensional scanning device emits laser or structured light and captures the light reflected back, measures the distance of the surface of the target person, and generates point cloud data representing the surface shape of the person. These initial point cloud data contain a large number of spatial coordinate points, each of which is a sampling point on the surface of the target person. Since the directly generated point cloud data is usually very large, in order to improve the efficiency of subsequent processing steps, the initial point cloud data is downsampled. The downsampling process involves reducing the number of points in the point cloud while retaining the important features of the original shape as much as possible. Through algorithms such as voxel grid filtering or random sampling, the burden of data processing is reduced without significantly losing model details. A graph structure is constructed based on the downsampled point cloud data, and the structural data of the graph structure is extracted. The spatial relationship between the point cloud data is understood and processed by converting the point cloud data into a graph structure, and using nodes and edges in the graph to represent the points in the point cloud and the relationship between them. This conversion enables the subsequent use of graph algorithms to analyze and process the point cloud data, such as identifying connected areas by traversing the graph or using the attributes of the graph to extract features. The structural data of the graph structure is input into the preset graph neural network to identify noise points. The graph neural network is a deep learning model that specializes in processing graph structure data. It can learn the high-dimensional feature representation of the nodes in the graph and perform various tasks based on these learned features, such as node classification, graph classification, etc. In this embodiment, the graph neural network is used to identify noise points, that is, those points that do not belong to the main body of the target person, which may be error points caused by interference from the scanning environment or other factors. The sampled point cloud data is denoised based on the noise point data, and those unnecessary noise points are removed to obtain the point cloud data of the three-dimensional form of the target person. The denoising process can be implemented by a variety of algorithms, such as statistical filtering, radius filtering, etc. These algorithms determine which points may be noise based on the distribution characteristics of the point cloud data and remove them. The denoised point cloud data is aligned to ensure that the point cloud data can accurately correspond to the real three-dimensional space of the target person. Data alignment, also known as point cloud registration, involves using an algorithm to adjust the direction and position of the point cloud so that multiple point cloud data sets can be correctly fused together in the same coordinate system.

[0041] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0042] (1) splitting the point cloud data into point cloud distribution areas to obtain multiple point cloud distribution areas;

[0043] (2) performing point cloud distribution density analysis on each point cloud distribution area respectively to obtain the point cloud distribution density of each point cloud distribution area;

[0044] (3) inputting the point cloud distribution density of each point cloud distribution area into a preset adaptive mesh reconstruction algorithm for mesh attribute matching to obtain mesh attribute data;

[0045] (4) Based on the grid attribute data, a digital human model of the target person is constructed in the metaverse by using a preset Poisson surface reconstruction algorithm to obtain a first digital human model;

[0046] (5) filling the void defects of the first digital human model to obtain a second digital human model;

[0047] (6) performing detail enhancement processing on the second digital human model to obtain a third digital human model;

[0048] (7) Perform multi-perspective fusion optimization on the third digital human model to obtain an initial digital human model.

[0049] Specifically, the point cloud data is split into multiple small areas by point cloud distribution area splitting. The point clouds in different areas may represent different parts of the character model, such as the head, arms or torso, and each part is different in geometric form and details. For example, by setting a certain spatial threshold or using a clustering algorithm, the point cloud can be divided into several areas according to the spatial proximity of each point in the point cloud. The point cloud distribution density is analyzed for each point cloud distribution area to evaluate the distribution density of the points in each area, that is, the point cloud density of each area. The point cloud density directly affects the degree of refinement and reconstruction difficulty in the subsequent reconstruction process, because high-density areas usually contain more detailed information. For example, the facial area of ​​the character model often requires a higher point cloud density to accurately capture facial features. The point cloud distribution density of each point cloud distribution area is input into the preset adaptive mesh reconstruction algorithm for mesh attribute matching. The role of the adaptive mesh reconstruction algorithm is to automatically adjust the size and density of the mesh based on the point cloud density to ensure that a finer mesh is used in areas with rich details and a larger mesh is used in areas with fewer details, thereby effectively balancing the capture of details and the consumption of computing resources in the reconstruction process. The mesh attribute data generated by this process contains information such as the size, shape and density of each area mesh. Based on the mesh attribute data, the preset Poisson surface reconstruction algorithm is used to construct a digital human model of the target person. The Poisson surface reconstruction algorithm is an efficient algorithm that can generate a continuous and smooth surface based on point cloud data and mesh attributes. The optimal form of the surface is solved by the mathematical Poisson equation, so that the reconstructed digital human model is both accurate and smooth. Thus, the first digital human model is obtained, which is a preliminary model reconstructed based on the original point cloud data and the optimized mesh attributes. The first digital human model may have holes and defects, which are caused by the incompleteness of the original point cloud data or the scanning blind area. Therefore, the model is then filled with holes and defects, and the algorithm is used to automatically detect and fill these holes to ensure the integrity of the model. After filling, the second digital human model is obtained. In order to further improve the quality of the model, the second digital human model is subjected to detail enhancement. The tiny textures and features on the surface of the model are processed, such as adding skin texture, clothing details, etc., to make the model look more realistic and detailed, and the third digital human model is obtained. The third digital human model is optimized by multi-view fusion. Through multi-perspective fusion optimization, the global consistency of the model is adjusted and optimized to ensure that the model is complete and natural from any angle, and finally the initial digital human model is obtained.

[0050] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0051] (1) Collecting texture images of the initial digital human to obtain model texture image data;

[0052] (2) extracting high-frequency details from the model texture image data to obtain high-frequency detail data;

[0053] (3) extracting low-frequency details from the model texture image data to obtain low-frequency detail data;

[0054] (4) Analyze the geometric features of the initial digital human to obtain a geometric feature set of the initial digital human;

[0055] (5) Based on the high-frequency detail data and the low-frequency detail data, the model curvature analysis is performed on the initial digital human to obtain the model curvature data;

[0056] (6) inputting the geometric feature set and the model curvature data into a preset texture mapping algorithm for texture attribute mapping to obtain target texture attributes;

[0057] (7) Based on the target texture attributes, the texture attributes of the initial digital human are configured to obtain the digital human model to be processed.

[0058] Specifically, texture image acquisition is performed on the initial digital human to collect actual image data used to cover the surface of the digital human model. These image data can be directly collected from the real world, such as photographing the target person's skin, clothing, etc. with a high-resolution camera, or selecting suitable textures from an image library. High-frequency and low-frequency detail extraction is performed on the model texture image data. High-frequency detail extraction focuses on small changes in the image, such as fine cracks in the skin texture or detailed texture of clothing. These details help to increase the realism of the digital person visually. At the same time, low-frequency detail extraction focuses on large-scale color gradients and larger texture patterns in the image. This information helps to build the basic appearance and overall feeling of the digital person. The initial digital human is subjected to geometric feature analysis to obtain the geometric feature set of the initial digital human. By analyzing the shape, curves and other geometric properties of the digital human model, important reference information is provided for subsequent texture mapping. The geometric feature set can help identify the key areas and specific features of the model, such as facial contours, muscle lines, etc. Based on the high-frequency detail data and low-frequency detail data, the initial digital human is subjected to model curvature analysis. The geometric changes on the model surface, including concavity, convexity, curvature, etc., are obtained to obtain model curvature data. These data will subsequently help ensure that the texture image can accurately fit the different curvatures of the model surface during the texture mapping process, because the display of the texture on the curved surface needs to be appropriately adjusted according to the curvature to prevent distortion caused by stretching or compression. The geometric feature set and model curvature data are input into the preset texture mapping algorithm. The algorithm automatically adjusts the texture image according to the geometric and curvature features of the model to accurately match the model surface and generate the target texture properties. The target texture properties include the specific placement, direction, and scaling of the texture on the model. Based on the target texture properties, the texture properties of the initial digital human are configured to obtain the digital human model to be processed. The texture image is accurately mapped to each part of the digital human model, including the adjustment and optimization of details to ensure the natural transition and seamless splicing of the texture.

[0059] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0060] (1) Collect the target person’s motion data and perform time series data matching on the motion data to obtain a time series data set;

[0061] (2) Extract key actions from action data based on the time series data set to obtain key action data;

[0062] (3) Scanning the key action data for action details to obtain an action detail data set;

[0063] (4) Perform detail coherence analysis on the action detail data set to obtain a detail coherence evaluation value;

[0064] (5) Optimizing the details of the action detail dataset based on the detail coherence evaluation value to obtain an optimized detail dataset;

[0065] (6) Based on the optimized detail data set, the model rendering of the digital human to be processed is optimized to obtain the target digital human model.

[0066] Specifically, the motion data of the target person is collected. The sensors installed on the key parts of the human body through the high-precision motion capture system record every movement and posture of the person, including walking, jumping, facial expressions, etc. These sensors can capture the slight changes in human body movements and convert them into digital signals to form an initial motion data set. Time series data matching is performed. The captured motion data are sorted and synchronized in chronological order to ensure the coherence and logic of the movements. The process of time series data matching includes analyzing the timestamp of each action and arranging them in the order of occurrence to form a continuous and smooth sequence of actions. Key actions are extracted based on the time series data set. Key actions refer to actions that play a connecting role in the entire action sequence. They are the key to understanding and restoring the target person's action pattern. For example, in a series of walking actions, the landing and starting of each step may be regarded as key actions. These key action data are extracted from complex action sequences through specific algorithms, such as action recognition and pattern analysis. After extracting the key action data, the action details are scanned for these data to obtain the action details data set. The subtle changes and features in each key action are analyzed, such as changes in hand posture and details of facial expressions. The action detail data set is analyzed for detail coherence to ensure the natural transition and consistency of the action details in the entire action sequence, and to avoid unnatural jumps or mutations between actions. By comparing the continuous action details in the action detail data set, the detail coherence evaluation value is calculated, which reflects the fluency and naturalness of the action sequence. Based on the detail coherence evaluation value, the action detail data set is optimized in detail. This includes adjusting the transition between actions, smoothing unnatural action breakpoints, and enhancing the natural fluency of actions to ensure that each action is presented in the most natural way. After optimization, an optimized action detail data set is obtained. Based on the optimized action detail data set, the model rendering optimization is performed on the digital human to be processed to obtain the final target digital human model. The optimized action data is applied to the digital human model, and through advanced rendering technology and algorithms, it is ensured that the model is not only accurate in appearance in a static state, but also can truly reflect the actions and expressions of the target person in dynamic performance.

[0067] In a specific embodiment, the process of performing the model rendering optimization step on the digital human to be processed may specifically include the following steps:

[0068] (1) constructing a lighting model for the digital human to be processed and obtaining a target lighting model;

[0069] (2) Analyze the illumination missing area of ​​the target illumination model through a ray tracing algorithm to obtain the illumination missing area;

[0070] (3) Enhance the illumination of the area lacking illumination to obtain an enhanced digital human model;

[0071] (4) Perform shadow edge detection on the enhanced digital human model to obtain the shadow outline;

[0072] (5) According to the target illumination model, shadow illumination parameters are matched on the shadow contour to obtain a shadow illumination parameter set;

[0073] (6) configuring parameters of the enhanced digital human model based on the shadow and illumination parameter set to obtain a configured digital human model;

[0074] (7) Based on the optimized detail data set, the configured digital human model is optimized for model rendering to obtain the target digital human model.

[0075] Specifically, a lighting model is constructed for the digital human to be processed. The illumination effect of light sources on objects in the real world is simulated, including the direction, intensity, color of the light source, and the reflection and refraction characteristics of light on different material surfaces. For example, when constructing a scene illumination model, it may be necessary to consider multiple light sources in the scene, such as sunlight, indoor lights, etc., and the effects of their interaction with digital characters. The target illumination model is analyzed through the ray tracing algorithm, especially for the identification of areas with missing illumination. The ray tracing algorithm is a technology that simulates the propagation of light and the interaction of objects. It can accurately calculate how light starts from the light source, passes through reflection, refraction and other processes, and finally reaches the observer's eyes. In this process, the algorithm will identify which areas are not fully illuminated, that is, areas with missing illumination. The illumination of the areas with missing illumination is enhanced. The lighting effect is improved by adjusting the illumination model or directly adding virtual light sources to these areas, so that the overall and local digital characters can be evenly and reasonably illuminated. Shadow edge detection is performed to identify and enhance the shadow boundary in the digital human model, that is, the shadow outline. Shadow is a dark area formed when light is blocked by an object. Its shape, size and edge clarity help to enhance the three-dimensional sense and depth of the scene. Through shadow edge detection, more accurate shadow outline information is obtained. According to the target illumination model, shadow illumination parameters are matched for the shadow outline. The relationship between the shadow outline and the illumination model is analyzed, and the shadow depth, blur and other attributes are adjusted to make the shadow effect consistent with the overall illumination effect. Through matching, a shadow illumination parameter set is obtained. Based on the shadow illumination parameter set, the parameters of the enhanced digital human model are configured, including adjusting the reflectivity, transparency and other material properties of the model surface to adapt to the illumination model and shadow effect. Based on the optimized detail data set, the configured digital human model is optimized for model rendering, and finally the target digital human model is obtained. All previous optimization results are integrated and applied to the digital human model, and rendering technology is used to achieve the final visual effect. This includes the application of advanced shading technology, texture mapping, detail enhancement and illumination effect adjustment to ensure that the model not only looks real in a static state, but also maintains a high degree of realism and coherence in dynamic demonstrations.

[0076] In a specific embodiment, the process of performing the model rendering optimization step on the configured digital human model based on the optimized detail data set may specifically include the following steps:

[0077] (1) Analyze the action types of the optimization detail data set to obtain an action type set;

[0078] (2) Based on the action type set, the optimized detail data set is grouped to obtain multiple groups of action detail data;

[0079] (3) performing action parameter analysis on each set of action detail data to obtain an action parameter set for each set of action detail data;

[0080] (4) Based on the action parameter data of each set of action detail data, the model rendering optimization is performed on the configured digital human model to obtain the target digital human model.

[0081] Specifically, the optimized detail data set is analyzed for action types to identify and classify a series of actions of the target person, such as walking, jumping, waving, etc. Based on the action type set, the optimized detail data set is grouped. All action data are classified according to the identified action types to form multiple groups of action detail data. Each group of data focuses on a specific type of action. Classification helps in the subsequent action parameter analysis and is also convenient for taking the most suitable rendering and optimization strategies for different types of actions. For example, for two completely different types of actions, their action details and required rendering techniques may be significantly different. Action parameter analysis is performed on each group of action detail data to obtain specific parameters of each action, such as the speed, amplitude, duration of the action, and the relationship between various parts of the body. The action parameter set is a quantitative representation of the action detail data, which provides key information for subsequent model rendering optimization. For example, when analyzing a jumping action, in addition to considering the height and duration of the jump, it is also necessary to analyze the angle of knee bending and the posture when landing. These parameters together determine the naturalness and realism of the action. Based on the action parameters of each group of action detail data, the configuration digital human model is optimized for model rendering to obtain the target digital human model. The action parameter set is applied to the rendering process of the digital human model, and the dynamic performance of the model is adjusted through advanced rendering technology and algorithms to ensure that each action is not only visually accurate, but also can truly reflect the action characteristics of the target person in the dynamic demonstration. This includes adjusting the model's skeletal animation, muscle deformation, and facial expressions to meet the specific requirements of the action.

[0082] The above describes the rendering method of the Metaverse Digital Human in the embodiment of the present application. The following describes the rendering device of the Metaverse Digital Human in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the rendering device of the Metaverse Digital Human includes:

[0083] The scanning module 201 is used to perform three-dimensional data scanning on the target person through a preset high-resolution three-dimensional scanning device to obtain point cloud data;

[0084] A construction module 202 is used to construct a digital human of the target person in the metaverse according to the point cloud data to obtain an initial digital human model;

[0085] Configuration module 203, used to configure texture attributes of the initial digital human to obtain a digital human model to be processed;

[0086] The optimization module 204 is used to collect the motion data of the target person, and perform model rendering optimization on the digital human to be processed according to the motion data to obtain a target digital human model.

[0087] Through the synergy of the above components, by using the preset high-resolution 3D scanning device to scan the 3D data of the target person, the subtle geometric features of the person can be captured, ensuring the high authenticity and accuracy of the digital human model. This provides a solid foundation for building a high-quality metaverse digital human. The quality of the point cloud data is effectively optimized by downsampling the initial point cloud data, building a graph structure, extracting structural data, identifying and denoising noise points, and aligning data. This not only improves the efficiency of data processing, but also ensures the clarity and accuracy of the geometric details of the final digital human model. Adopting the adaptive mesh reconstruction algorithm and the Poisson surface reconstruction algorithm, combined with detail enhancement processing and multi-view fusion optimization, the mesh can be automatically adjusted according to the distribution of the point cloud data, the void defects can be filled, and the model details can be enhanced. This series of optimization measures greatly improves the visual authenticity and aesthetics of the digital human model. Through the acquisition of texture images, detail extraction, geometric feature analysis and texture mapping algorithms, the texture properties of the digital human model can be accurately configured. In particular, the combination of high-frequency and low-frequency detail data, as well as the model curvature analysis, ensures the natural fit of the texture and the consistency of the visual effect. The target person's motion data is collected and analyzed. Through timing matching, key motion extraction, motion detail scanning and coherence analysis, the naturalness and smoothness of the motion details are optimized. This ensures that the digital human's motion performance in the metaverse is both realistic and natural. The target lighting model is constructed and the lighting effect is optimized through ray tracing technology. Combined with shadow contour detection and lighting parameter matching, the three-dimensional sense and presence of the digital human model are further enhanced. Through model rendering optimization, it is ensured that the digital human model shows highly realistic and attractive visual effects under various lighting conditions.

[0088] The present application also provides a rendering device for a metaverse digital human, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the rendering method for a metaverse digital human in the above-mentioned embodiments.

[0089] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the method for rendering a metaverse digital human.

[0090] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0091] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0092] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for rendering a digital human in the metaverse, characterized in that: The method for rendering a metaverse digital human comprises: The target person is scanned in three dimensions by a preset high-resolution three-dimensional scanning device to obtain point cloud data; Constructing a digital human of the target person in the metaverse according to the point cloud data to obtain an initial digital human model; The initial digital human is configured with texture attributes to obtain a digital human model to be processed; specifically comprising: collecting texture images of the initial digital human to obtain model texture image data; extracting high-frequency details from the model texture image data to obtain high-frequency detail data; extracting low-frequency details from the model texture image data to obtain low-frequency detail data; performing geometric feature analysis on the initial digital human to obtain a geometric feature set of the initial digital human; performing model curvature analysis on the initial digital human based on the high-frequency detail data and the low-frequency detail data to obtain model curvature data; inputting the geometric feature set and the model curvature data into a preset texture mapping algorithm to perform texture attribute mapping to obtain target texture attributes; and configuring texture attributes of the initial digital human based on the target texture attributes to obtain a digital human model to be processed; The action data of the target person is collected, and model rendering optimization is performed on the digital human to be processed according to the action data to obtain a target digital human model; specifically comprising: collecting the action data of the target person, and performing time series data matching on the action data to obtain a time series data set; extracting key actions from the action data based on the time series data set to obtain key action data; scanning the action details of the key action data to obtain an action detail data set; performing detail coherence analysis on the action detail data set to obtain a detail coherence evaluation value; performing detail optimization on the action detail data set based on the detail coherence evaluation value to obtain an optimized detail data set; performing model rendering optimization on the digital human to be processed based on the optimized detail data set to obtain a target digital human model.

2. The method for rendering a digital human in the Metaverse according to claim 1, characterized in that: The three-dimensional data scanning of the target person is performed by a preset high-resolution three-dimensional scanning device to obtain point cloud data, including: Performing three-dimensional data scanning on the target person by means of the high-resolution three-dimensional scanning device to obtain initial point cloud data; Downsampling the initial point cloud data to obtain sampled point cloud data; Constructing a graph structure according to the sampling point cloud data, and extracting structural data of the graph structure; Inputting the structural data into a preset graph neural network to perform noise point recognition to obtain noise point data; De-noising the sampling point cloud data based on the noise point data to obtain de-noised point cloud data; Performing data alignment processing on the denoised point cloud data to obtain the point cloud data.

3. The method for rendering a digital human of the Metaverse according to claim 1, characterized in that: The step of constructing a digital human of the target person in the metaverse according to the point cloud data to obtain an initial digital human model includes: Splitting the point cloud data into point cloud distribution areas to obtain a plurality of point cloud distribution areas; Performing point cloud distribution density analysis on each of the point cloud distribution areas respectively to obtain the point cloud distribution density of each of the point cloud distribution areas; Inputting the point cloud distribution density of each point cloud distribution area into a preset adaptive grid reconstruction algorithm for grid attribute matching to obtain grid attribute data; Based on the grid attribute data, constructing a digital human model of the target person in the metaverse by using a preset Poisson surface reconstruction algorithm to obtain a first digital human model; Filling the first digital human model with void defects to obtain a second digital human model; performing detail enhancement processing on the second digital human model to obtain a third digital human model; The third digital human model is subjected to multi-view fusion optimization to obtain the initial digital human model.

4. The method for rendering a digital human of the Metaverse according to claim 1, characterized in that: The step of performing model rendering optimization on the digital human to be processed based on the optimized detail data set to obtain a target digital human model includes: Constructing a lighting model for the digital human to be processed to obtain a target lighting model; Performing illumination missing area analysis on the target illumination model by using a ray tracing algorithm to obtain illumination missing areas; Performing illumination enhancement on the illumination-deficient area to obtain an enhanced digital human model; Performing shadow edge detection on the enhanced digital human model to obtain a shadow outline; According to the target illumination model, shadow illumination parameter matching is performed on the shadow outline to obtain a shadow illumination parameter set; Performing parameter configuration on the enhanced digital human model based on the shadow illumination parameter set to obtain a configured digital human model; Model rendering optimization is performed on the configured digital human model based on the optimization detail data set to obtain the target digital human model.

5. The method for rendering a digital human of the Metaverse according to claim 4, characterized in that: The performing model rendering optimization on the configured digital human model based on the optimized detail data set to obtain the target digital human model includes: Performing action type analysis on the optimization detail data set to obtain an action type set; Based on the action type set, grouping the optimized detail data set to obtain multiple groups of action detail data; Performing action parameter analysis on each group of action detail data to obtain an action parameter set for each group of action detail data; The configured digital human model is subjected to model rendering optimization based on the action parameter data of each group of the action detail data to obtain the target digital human model.

6. A rendering device for a metaverse digital human, characterized in that: The rendering device of the Metaverse Digital Human includes: A scanning module is used to perform three-dimensional data scanning on a target person through a preset high-resolution three-dimensional scanning device to obtain point cloud data; A construction module, used to construct a digital human of the target person in the metaverse according to the point cloud data to obtain an initial digital human model; A configuration module is used to configure the texture attributes of the initial digital human to obtain a digital human model to be processed; specifically comprising: performing texture image acquisition on the initial digital human to obtain model texture image data; performing high-frequency detail extraction on the model texture image data to obtain high-frequency detail data; performing low-frequency detail extraction on the model texture image data to obtain low-frequency detail data; performing geometric feature analysis on the initial digital human to obtain a geometric feature set of the initial digital human; performing model curvature analysis on the initial digital human based on the high-frequency detail data and the low-frequency detail data to obtain model curvature data; inputting the geometric feature set and the model curvature data into a preset texture mapping algorithm to perform texture attribute mapping to obtain target texture attributes; performing texture attribute configuration on the initial digital human based on the target texture attributes to obtain a digital human model to be processed; The optimization module is used to collect the action data of the target person, and perform model rendering optimization on the digital human to be processed according to the action data to obtain the target digital human model; specifically includes: collecting the action data of the target person, and performing time series data matching on the action data to obtain a time series data set; extracting key actions from the action data based on the time series data set to obtain key action data; scanning the action details of the key action data to obtain an action detail data set; performing detail coherence analysis on the action detail data set to obtain a detail coherence evaluation value; performing detail optimization on the action detail data set based on the detail coherence evaluation value to obtain an optimized detail data set; performing model rendering optimization on the digital human to be processed based on the optimized detail data set to obtain the target digital human model.

7. A rendering device for a metaverse digital human, characterized in that: The Metaverse Digital Human rendering device comprises: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory so that the rendering device of the Metaverse digital human executes the rendering method of the Metaverse digital human as described in any one of claims 1-5.

8. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the method for rendering a metaverse digital human as described in any one of claims 1-5 is implemented.

Images