Modularized engine-based meta universe scene rapid construction method and system

Abstract

The invention provides a meta-universe scene rapid construction method and system based on a modular engine, and relates to the technical field of meta universe, and the method comprises the steps: obtaining a creation instruction of a user for a meta-universe scene, and calling a corresponding scene rendering module from a preset modular engine library according to scene type information; performing feature extraction and scene layering on scene environment parameters based on a scene intelligent analysis model of deep learning, and analyzing the scene environment parameters into a plurality of scene function layers with a progressive hierarchical relationship; and calling a physical engine module, an illumination rendering module and an environment special effect module to construct a scene. Through the modular engine and the intelligent analysis model, rapid establishment of the meta-universe scene is realized, the scene establishment efficiency is improved, and the scene reality sense and the interaction experience are enhanced.

Description

Technical Field

[0001] The present invention relates to metaverse technology, and in particular to a method and system for quickly building a metaverse scene based on a modular engine. Background Art

[0002] With the rapid development of metaverse technology, the construction and rendering of virtual scenes have become an important research direction. As an important carrier for users to interact with virtual objects, the construction quality of metaverse scenes directly affects the user's immersive experience. At present, the construction of metaverse scenes mainly relies on traditional 3D modeling software and game engines. Developers need to manually complete scene modeling, material setting, lighting rendering and other tasks. This traditional scene construction method can no longer meet the needs of the current rapid development of the metaverse.

[0003] The main problems existing in the prior art are as follows: Traditional scene construction methods lack modular and standardized technical frameworks. Developers need to repeat the construction of similar scenes, resulting in low development efficiency and difficulty in ensuring the consistency of scene quality. The various links in the scene construction process are often fragmented and lack a unified management and scheduling mechanism.

[0004] Existing scene rendering technologies fail to fully consider the hierarchical structure of the scene and the environmental constraints, resulting in unrealistic rendering effects. Especially in complex scenes, the lighting interactions between different levels and environmental effects are difficult to simulate accurately, affecting the realism and immersion of the scene. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for quickly building a metaverse scene based on a modular engine, which can solve the problems in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for quickly building a metaverse scene based on a modular engine, comprising: Obtaining a user's creation instruction for a metaverse scene, wherein the creation instruction includes scene type information and scene environment parameters; According to the scene type information, a corresponding scene rendering module is called from a preset modular engine library, wherein the scene rendering module includes a physical engine module, a lighting rendering module, and an environmental special effect module; Based on the deep learning scene intelligent analysis model, the scene environment parameters are feature extracted and the scene is layered, and the scene environment parameters are parsed into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints; a progressive hierarchical relationship is formed between the scene function layers, and the environmental constraints of the upper scene function layer will optimize and adjust the rendering results of the lower scene function layer; Calling the physical engine module to construct a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and embedding corresponding physical property parameters in each three-dimensional mesh model; Calling the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layers; The environmental special effects module is called to add corresponding environmental special effects to each of the scene function layers according to the scene environment parameters.

[0007] According to the scene type information, calling a corresponding scene rendering module from a preset modular engine library includes: Receiving scene type information, and encoding the scene type information through a pre-trained BERT model to obtain a scene feature vector; A candidate rendering module is obtained from the modular engine library, and the matching degree between the candidate rendering module and the scene feature vector is calculated. The matching degree is calculated as follows: ; in, M Represents a set of candidate rendering modules, S represents the scene feature vector set, α represents the balance factor, n represents the feature dimension, w i represents the weight coefficient of the i-th dimension feature, sim(m i ,s i ) represents the similarity between the candidate rendering module of the i-th dimension and the feature vector of the i-th dimension scene, β represents the balance factor, C(M) Represents the computational complexity evaluation value corresponding to the candidate rendering module set, C() represents the complexity evaluation function; The candidate rendering module with the highest matching degree is selected as the scene rendering module corresponding to the scene type information.

[0008] Based on the deep learning scene intelligent analysis model, the scene environment parameters are feature extracted and the scene is layered, and the scene environment parameters are parsed into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints, including: Extracting geometric features and material features from the scene environment parameters respectively through a multimodal feature extraction network, wherein the geometric features are extracted from a scene point cloud set through a PointNet++ network, and the material features are extracted from a material map through a multi-scale convolutional neural network; A multi-head attention mechanism is used to fuse the geometric features and the material features, the geometric features are used as query vectors, the material features are used as key vectors, and the concatenation result of the geometric features and the material features is used as a value vector to generate fused features; Performing hierarchical analysis on the fused features, dividing the scene into multiple functional layers, and establishing a hierarchical relationship between the functional layers; Generate geometric information, material information and interaction rules of each functional layer through a multi-layer perceptron respectively; A two-layer constraint optimization model is constructed, which includes local constraints within a functional layer and global constraints between functional layers, and constraint optimization is achieved by minimizing a weighted combination of constraint expected deviations and constraint differences between adjacent functional layers.

[0009] A two-layer constraint optimization model is constructed, wherein the two-layer constraint optimization model includes local constraints within a functional layer and global constraints between functional layers, and constraint optimization is achieved by minimizing a weighted combination of constraint expected deviations and constraint differences between adjacent functional layers, including: Constructing local constraint features of the scene function layer, the local constraint features include geometric constraint features and material constraint features, wherein the geometric constraint features include position continuity constraints, normal vector consistency constraints and boundary continuity constraints, and the material constraint features include texture transition constraints, roughness continuity constraints and reflectivity consistency constraints; Constructing global constraint features of scene function layers, the global constraint features include inter-layer topological constraints and physical constraints, wherein the inter-layer topological constraints calculate the constraint relationship between adjacent functional layers through inter-layer association weights and topological relationship measurement functions, and the physical constraints include gravity constraints, collision detection constraints, and structural stability constraints; Constructing a multi-level optimization objective function based on the local constraint features and the global constraint features, taking a weighted combination of the geometric constraints and material constraints of the local constraint features as a local optimization objective, and taking a weighted combination of the topological constraints and physical constraints of the global constraint features as a global optimization objective; Adopting an adaptive weight adjustment mechanism to dynamically balance the local optimization target and the global optimization target, the adaptive weight is adjusted exponentially based on the ratio of the local optimization target value to the global optimization target value; Iteratively solving the multi-level optimization objective function by an alternating direction multiplier method until the difference between the optimization results of two adjacent iterations is less than a preset deviation threshold; The local constraint satisfaction and the global constraint satisfaction of each functional layer are calculated, and the local constraint satisfaction and the global constraint satisfaction are both calculated using a Gaussian evaluation function.

[0010] Calling the physical engine module, constructing a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and implanting corresponding physical attribute parameters in each three-dimensional mesh model includes: Constructing an adaptive tetrahedral mesh according to the geometric information, performing quality evaluation and mesh unit optimization on the tetrahedral mesh by mesh unit volume ratio, mesh unit dihedral angle and mesh unit side length ratio, and generating a high-quality tetrahedral mesh; Calculating physical property parameters based on the material information, respectively calculating mass density, elastic modulus, Poisson's ratio and friction coefficient of the scene function layer through a material-physical mapping network, and establishing a corresponding relationship between the material information and the physical property parameters; The physical property parameters are distributed to each mesh unit of the high-quality tetrahedral mesh to generate a three-dimensional physical mesh model with physical properties.

[0011] Calling the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layer includes: The hierarchical relationship includes the spatial position relationship and occlusion relationship between the functional layers; Based on the hierarchical relationship, a hierarchical illumination propagation tree is constructed, wherein the nodes of the hierarchical illumination propagation tree represent scene function layers, the connecting edges between the nodes represent illumination propagation paths, and an illumination attenuation weight is assigned to each illumination propagation path according to the spatial position relationship and the occlusion relationship; According to the structure of the hierarchical illumination propagation tree, direct illumination and indirect illumination of each of the three-dimensional grid models are calculated to generate an illumination interaction matrix, wherein the illumination interaction matrix records the light energy transfer relationship between different functional layers; A depth map of each of the three-dimensional mesh models is generated based on the illumination interaction matrix, and a shadow mapping matrix is ​​calculated in combination with the illumination attenuation weights, and the illumination distribution of each functional layer is dynamically updated and optimized according to the shadow mapping matrix.

[0012] A second aspect of an embodiment of the present invention provides a metaverse scene rapid construction system based on a modular engine, including: The first unit is used to obtain a user's creation instruction for a metaverse scene, wherein the creation instruction includes scene type information and scene environment parameters; The second unit is used to call a corresponding scene rendering module from a preset modular engine library according to the scene type information, wherein the scene rendering module includes a physical engine module, a lighting rendering module and an environmental special effect module; The third unit is used for extracting features and stratifying the scene environment parameters based on a deep learning scene intelligent analysis model, parsing the scene environment parameters into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints; a progressive hierarchical relationship is formed between the scene function layers, and the environmental constraints of the upper scene function layer will optimize and adjust the rendering results of the lower scene function layer; The fourth unit is used to call the physical engine module, construct a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and implant corresponding physical property parameters in each three-dimensional mesh model; A fifth unit is used to call the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layers; The sixth unit is used to call the environmental special effects module and add corresponding environmental special effects to each of the scene function layers according to the scene environment parameters.

[0013] A third aspect of the embodiments of the present invention An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0014] According to a fourth aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0015] The beneficial effects of this application are as follows: The present invention can realize efficient creation and rendering of scenes through a rapid construction method of metaverse scenes based on a modular engine, improve the automation of scene construction and the realism of rendering effects, which is specifically reflected in the following aspects: Through the combined call of the preset modular engine library and scene rendering module, the standardization and modularization of the scene construction process is achieved, which greatly improves the efficiency of scene creation, reduces repetitive development work, and makes the scene construction process more convenient and flexible.

[0016] The scene intelligent analysis model based on deep learning is used to process the scene in layers, and a progressive hierarchical relationship is established, so that the various functional layers of the scene can be optimized and complemented, which improves the coordination and realism of the overall rendering effect, and also facilitates the subsequent scene maintenance and updating.

[0017] Through the collaborative work of the physical engine module, lighting rendering module and environmental special effects module, each functional layer in the scene is endowed with real physical properties, lighting effects and environmental atmosphere, greatly enhancing the user's immersion and interactive experience in the metaverse scene, making the scene more vivid and realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a method for quickly building a metaverse scene based on a modular engine according to an embodiment of the present invention; Figure 2 A logical diagram for building a two-level constrained optimization model. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0021] Figure 1 Schematic diagram of the process of the method for quickly building a metaverse scene based on a modular engine according to an embodiment of the present invention. Figure 1 As shown, the method includes: Obtaining a user's creation instruction for a metaverse scene, wherein the creation instruction includes scene type information and scene environment parameters; According to the scene type information, a corresponding scene rendering module is called from a preset modular engine library, wherein the scene rendering module includes a physical engine module, a lighting rendering module, and an environmental special effect module; Based on the deep learning scene intelligent analysis model, the scene environment parameters are feature extracted and the scene is layered, and the scene environment parameters are parsed into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints; a progressive hierarchical relationship is formed between the scene function layers, and the environmental constraints of the upper scene function layer will optimize and adjust the rendering results of the lower scene function layer; Calling the physical engine module to construct a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and embedding corresponding physical property parameters in each three-dimensional mesh model; Calling the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layers; The environmental special effects module is called to add corresponding environmental special effects to each of the scene function layers according to the scene environment parameters.

[0022] Exemplarily, the system obtains the user's creation instruction for the metaverse scene, and the creation instruction includes scene type information and scene environment parameters. The scene type information can be a variety of preset scene categories, such as urban scenes, natural scenes, science fiction scenes, historical scenes, etc.; scene environment parameters include but are not limited to time settings (such as daytime, dusk, night), weather conditions (such as sunny days, rainy days, snowy days), geographical environments (such as plains, mountains, oceans), and scene scales. The system provides a graphical interface, through which users can select scene types and adjust various environmental parameters, or input creation instructions in the form of natural language descriptions, and the system converts them into standardized scene type information and environmental parameters through natural language processing technology.

[0023] According to the scene type information obtained, the system calls the corresponding scene rendering module from the preset modular engine library. The modular engine library contains a variety of rendering modules optimized for different scene types. Each scene type corresponds to a specific combination of physical engine modules, lighting rendering modules, and environmental special effects modules. The physical engine module is responsible for simulating the physical behavior of objects in the scene, including physical rules such as gravity, collision detection, and fluid mechanics; the lighting rendering module is responsible for processing the lighting effects in the scene, including global illumination, local illumination, reflection, refraction and other optical phenomena; the environmental special effects module is responsible for generating special effects in the scene, such as fog, rain, snow, fire, smoke, etc. The system automatically selects the module combination that best suits the current scene type through the preset mapping relationship table between scene types and rendering modules, and supports users to manually adjust module selection to meet specific needs.

[0024] The system uses a scene intelligent analysis model based on deep learning to extract features and stratify scene environment parameters. By learning from a large amount of existing metaverse scene data, it can identify key features in scene environment parameters and stratify them reasonably. The analysis process first converts the scene environment parameters into high-dimensional feature vectors, then performs feature mapping and dimensionality reduction through a multi-layer perceptron, and finally parses the scene environment parameters into multiple scene function layers. Each scene function layer includes corresponding geometric information (such as terrain height map, building distribution map), material information (such as surface texture, building material), interaction rules (such as the behavior definition of interactive objects), and environmental constraints (such as light intensity restrictions, physical behavior constraints).

[0025] These scene function layers form a progressive hierarchical relationship, generally starting from the basic terrain layer, and then constructing the vegetation layer, building layer, decoration layer, special effects layer, etc. in sequence. The environmental constraints of the upper scene function layer will optimize and adjust the rendering results of the lower scene function layer. For example, the shadow projection of the building layer will affect the lighting effect of the surface layer; the rain and snow effects of the special effects layer will change the material performance of the surface layer and the building layer. The system handles the influence relationship between each layer in a recursive manner to ensure the consistency and realism of the rendering results of each layer. During the layering process, the system will also automatically adjust the number of layers according to the complexity of the scene. For simple scenes, only 3-4 functional layers may be required, while complex scenes may require more than 10 functional layers for fine control.

[0026] The system calls the physical engine module to build a three-dimensional mesh model based on the geometric information and material information of each scene function layer. The physical engine module first converts the geometric information of each scene function layer into point cloud data, and then generates a preliminary three-dimensional mesh through a triangulation algorithm. For complex geometric bodies, the system uses subdivision surface technology to further optimize the mesh quality and ensure the smoothness of the model surface. The material information is applied to the mesh model through UV mapping technology to achieve accurate texture correspondence. After the mesh model is built, the system will implant the corresponding physical property parameters in each three-dimensional mesh model, including mass, density, elastic coefficient, friction coefficient, etc. These physical property parameters are automatically matched according to the material information. For example, metal materials will be given higher density and lower elastic coefficient, while wood materials will be given the opposite. For special materials, the system provides a manual adjustment interface for physical property parameters, allowing users to accurately control the physical behavior characteristics of objects in the scene.

[0027] The system calls the lighting rendering module to perform lighting calculation and shadow mapping for each three-dimensional mesh model based on the hierarchical relationship of the scene function layer. The lighting rendering module first determines the main light source properties (such as sunlight or moonlight) according to the time setting and weather conditions in the scene environment parameters, including the light source position, intensity, color, etc. Then a layered rendering strategy is adopted to perform lighting calculation layer by layer starting from the bottom functional layer. For each layer, the system applies a real-time global illumination algorithm to calculate the effects of direct and indirect lighting, and generates a light map to store the calculation results to improve rendering efficiency. Shadow mapping uses cascade shadow mapping technology to automatically adjust the shadow resolution according to the viewing distance, using high-precision shadows for close distances and low-precision shadows for far distances to balance visual effects and performance consumption. In the multi-layer rendering process, the shadows of the upper objects will be correctly projected onto the lower objects to achieve real lighting effects in complex scenes. The system also supports physically based rendering technology to simulate the reflection, refraction and scattering characteristics of different materials to light, and enhance the realism of rendering.

[0028] The system call environment special effects module adds corresponding environmental special effects to each scene function layer according to the scene environment parameters. The environmental special effects module includes a variety of preset special effects templates, such as fog systems, precipitation systems, wind systems, day-night cycle systems, etc. The system automatically selects a suitable special effects combination according to the weather conditions, time settings, etc. in the scene environment parameters. For example, in a rainy scene, the system will enable the precipitation system to generate raindrop particles and at the same time enable the wet ground effect to enhance the reflective characteristics of the material. The special effects generation uses a physics-based particle system that can simulate the real behavior of particles in the environment, such as raindrop falling, water accumulation formation, fog diffusion, etc. In addition, the environmental special effects also include atmosphere rendering, which creates a specific scene atmosphere by adjusting the overall tone, post-processing filters, etc. Each scene function layer receives different environmental special effects according to its characteristics. For example, the water surface layer will produce a rippling reaction to the raindrop special effects, and the vegetation layer will produce a swaying reaction to the wind special effects. The system ensures the consistency and coordination of the special effects performance with the overall scene style through the adaptive adjustment of the special effects parameters.

[0029] In an alternative embodiment, Invoking the corresponding scene rendering module from the preset modular engine library according to the scene type information includes: Receiving the scene type information and encoding the scene type information through a pre-trained BERT model to obtain a scene feature vector; Obtaining candidate rendering modules from the modular engine library and calculating the matching degree between the candidate rendering modules and the scene feature vector. The calculation formula for the matching degree is: ; Where M represents the candidate rendering module set, S represents the scene feature vector set, α represents the balance factor, n represents the feature dimension, w i represents the weight coefficient of the i-th dimension feature, sim(m i ,s i ) represents the similarity between the i-th dimension candidate rendering module and the i-th dimension scene feature vector, β represents the balance factor, C(M) represents the computational complexity evaluation value corresponding to the candidate rendering module set, C() represents the complexity evaluation function; Selecting the candidate rendering module with the highest matching degree as the scene rendering module corresponding to the scene type information.

[0030] Exemplarily, the scene type information may be natural language descriptions such as "futuristic city square", "ancient oriental palace", etc., or a combination of tags selected from predefined categories. For example, the user may input "a foggy medieval European town, rainy day, dusk" as the scene type information.

[0031] After receiving the scene type information, the system encodes it through a pre-trained BERT model. This BERT model adopts the basic BERT-base architecture, contains 12 layers of Transformer encoders, each layer has 12 attention heads, and the hidden layer dimension is 768. The model has undergone two-stage training: first, it is pre-trained on a general corpus, and then it is fine-tuned for domain adaptation using about 500,000 metaverse scene description corpora. The fine-tuning process uses the Adam optimizer with a learning rate of 2e-5, a batch size of 32, and is trained for 10 epochs. After inputting the scene type information into the model, the system extracts the hidden state corresponding to the CLS token from the last layer, and then maps it to a 128-dimensional scene feature vector through a linear layer. For the above example, the obtained scene feature vector may have relatively high activation values in dimensions such as "medieval", "Europe", "fog", "rainy day", and "dusk", which are 0.82, 0.79, 0.65, 0.88, and 0.71 respectively.

[0032] After obtaining the scene feature vector, the system retrieves candidate rendering modules from the modular engine library. The modular engine library stores various pre-developed rendering modules, and each module contains its functional feature description, performance parameters, and applicable scene tags. The system first performs a preliminary screening through keyword matching and filters out 20 potentially relevant candidate modules from 500 rendering modules in the library. These candidate modules include "Medieval European architecture renderer", "Rainy day special effect engine", "Fog simulator", "Dusk lighting system", etc.

[0033] The system calculates the matching degree between the candidate rendering modules and the scene feature vector. The matching degree calculation comprehensively considers two factors: functional matching and computational complexity. For functional matching, the system calculates the similarity of each dimension of the feature and performs weighted summation. In the specific process, the system sets weight coefficients for each of the 128 dimensions of the feature vector. Important dimensions such as scene style and main environmental elements have higher weights (0.08 - 0.12), and secondary dimensions such as detail special effects have lower weights (0.01 - 0.05). The similarity calculation uses the cosine similarity method, and the value range is from -1 to 1. The closer to 1, the more similar it is.

[0034] For example, for the "Medieval European Architecture Renderer" module, its eigenvalue in the "Medieval" dimension is 0.95, and the similarity calculated with the scene feature vector of 0.82 is 0.96; the eigenvalue in the "Europe" dimension is 0.88, and the similarity calculated with the scene feature of 0.79 is 0.94. Assuming that the weights of these two dimensions are 0.10 and 0.09 respectively, the weighted similarity contribution of these two dimensions is 0.10×0.96+0.09×0.94=0.1786. Similar calculations are performed for all 128 dimensions and summed up, and the total functional matching score of the module is 0.76.

[0035] In terms of computational complexity, the system uses a pre-trained performance prediction model to evaluate the resource consumption of each rendering module. The model is trained based on historical operating data, with module parameters (such as the number of polygons, texture resolution, lighting algorithm complexity, etc.) as input, and expected CPU usage, GPU usage, memory usage, and frame rate impact as output. These indicators are weighted and synthesized to obtain a standardized complexity evaluation value, ranging from 0 to 1, with larger values ​​indicating higher complexity. For example, the complexity evaluation value of the "Medieval European Architecture Renderer" is 0.65, indicating medium to high resource consumption.

[0036] The final matching degree is calculated by balancing the function matching score and the complexity evaluation value. The balance factor α is used to adjust the importance of function matching, usually with a value of 0.7-0.9; the balance factor β is used to adjust the influence of complexity, usually with a value of 0.1-0.3. In this example, α=0.8, β=0.2, then the final matching degree of "Medieval European Architecture Renderer" is 0.8×0.76-0.2×0.65=0.608-0.13=0.478. After calculating the matching degree of all candidate rendering modules, the system selects the module with the highest matching degree as the final result.

[0037] In an optional embodiment, Based on the deep learning scene intelligent analysis model, the scene environment parameters are feature extracted and the scene is layered, and the scene environment parameters are parsed into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints, including: Extracting geometric features and material features from the scene environment parameters respectively through a multimodal feature extraction network, wherein the geometric features are extracted from a scene point cloud set through a PointNet++ network, and the material features are extracted from a material map through a multi-scale convolutional neural network; A multi-head attention mechanism is used to fuse the geometric features and the material features, the geometric features are used as query vectors, the material features are used as key vectors, and the concatenation result of the geometric features and the material features is used as a value vector to generate fused features; Performing hierarchical analysis on the fused features, dividing the scene into multiple functional layers, and establishing a hierarchical relationship between the functional layers; Generate geometric information, material information and interaction rules of each functional layer through a multi-layer perceptron respectively; A two-layer constraint optimization model is constructed, which includes local constraints within a functional layer and global constraints between functional layers, and constraint optimization is achieved by minimizing a weighted combination of constraint expected deviations and constraint differences between adjacent functional layers.

[0038] Figure 2 The following is a logical diagram for constructing a two-layer constrained optimization model. Exemplarily, the geometric features and material features in the scene environment parameters are extracted respectively through a multimodal feature extraction network. For geometric features, the system uses the PointNet++ network to extract from the scene point cloud set. In specific implementation, the input point cloud data usually contains millions of sampling points, each of which contains three-dimensional coordinates and normal vector information. The system first downsamples the point cloud, and uses the voxel downsampling method to reduce the point cloud density to about 1000 points per cubic meter, maintaining the geometric structure while improving processing efficiency. The downsampled point cloud is processed by the PointNet++ network, which contains 3 hierarchical sampling modules, each of which contains a sampling layer, a grouping layer, and a PointNet layer. The first layer samples 1024 points, the second layer samples 256 points, and the third layer samples 64 points, capturing the local and global geometric features of the point cloud through layer-by-layer abstraction. The network output is a 512-dimensional geometric feature vector. For example, when processing a city square scene, the extracted geometric feature vector may contain representations of geometric elements such as a plane area (eigenvalue 0.85), a stepped structure (eigenvalue 0.72), and a circular fountain (eigenvalue 0.68).

[0039] For material features, the system uses a multi-scale convolutional neural network to extract from material maps. Material maps are usually RGB images with a resolution of 2048×2048, containing information such as color, roughness, metalness, and normals. The system uses a multi-branch network based on ResNet-50, and each branch processes material information at different scales. The first branch processes global materials (2048×2048→256×256), the second branch processes medium-scale materials (512×512), and the third branch processes microscopic material details (128×128). The features of the three branches are fused through the attention mechanism to generate a 256-dimensional material feature vector. In the city square example, the extracted material features may include material attributes such as stone texture (eigenvalue 0.91), metal decoration (eigenvalue 0.43), and vegetation cover (eigenvalue 0.37).

[0040] The model uses a multi-head attention mechanism to fuse geometric features and material features. The specific implementation uses an 8-head attention mechanism, and the dimension of each attention head is 64. In the attention calculation, the geometric feature vector (512 dimensions) is used as the query vector, the material feature vector (256 dimensions) is used as the key vector, and the concatenation of the two (768 dimensions) is used as the value vector. The query vector and the key vector are first mapped to the same dimension (512 dimensions) through a linear projection layer, and then the attention weights are calculated. In the city square example, the attention weight of the stone material with the plane area is 0.82, and the weight with the ladder is 0.79, which indicates that the stone material is mainly used in these geometric structures. The output of the multi-head attention is residually connected and layer normalized to generate a 768-dimensional fused feature vector, which contains the associated information of both geometry and material.

[0041] The model performs hierarchical analysis of the fused features and divides the scene into multiple functional layers. The system uses a hierarchical clustering algorithm to cluster the fused feature vectors into 5-10 functional layers based on feature similarity. In the city square example, the system divides the scene into five functional layers: basic terrain layer, building structure layer, decorative facilities layer, vegetation layer, and special effects layer. After clustering, the system uses graph neural networks to establish hierarchical relationships between functional layers and constructs a directed acyclic graph to represent hierarchical dependencies. For example, the basic terrain layer is at the bottom layer, the building structure layer and vegetation layer depend on the terrain layer, the decorative facilities layer depends on the building layer, and the special effects layer is at the top layer and depends on all other layers.

[0042] After the stratification is completed, the model generates the geometric information, material information and interaction rules of each functional layer through a multi-layer perceptron. The geometric information generation uses a 5-layer fully connected network, with the input being the fusion feature of the layer (768 dimensions), and outputs the geometric parameters of the functional layer, such as vertex coordinates and patch index, through a hidden layer with an inter-layer dimension of [768, 512, 256, 128, 64]. The material information generation uses a 3-layer fully connected network with a hidden layer dimension of [768, 384, 192], and outputs the material parameters of the layer, such as base color, roughness, and metalness. The interaction rule generation uses a 4-layer fully connected network with a hidden layer dimension of [768, 384, 192, 96], and outputs the interaction parameters of the layer, including collision properties, interactive object identification, etc. Taking the building structure layer as an example, the generated geometric information includes about 20,000 vertices and 15,000 faces; the material information includes sandstone material (basic color RGB value [0.82, 0.78, 0.65], roughness 0.72); the interaction rules include walkable area markings, climbable wall markings, etc.

[0043] The model constructs a two-layer constraint optimization model to generate environmental constraints. The optimization model includes local constraints within the functional layer and global constraints between functional layers. Local constraints focus on the consistency within a single functional layer, such as the physical stability of the building structure, the realism of the material, etc.; global constraints focus on the coordination between functional layers, such as lighting consistency, physical interaction compatibility, etc. The system minimizes the weighted combination of constraint expected deviations and constraints differences between adjacent functional layers through an iterative optimization algorithm. During the optimization process, the local constraint weight is set to 0.7 and the global constraint weight is set to 0.3. Taking the lighting constraint as an example, the system ensures that the light source of the special effects layer can correctly affect all underlying structures, produce reasonable shadows on the surface of the building layer (deviation is less than 0.15), and keep the lighting transition with the terrain layer natural (the difference between layers is less than 0.2).

[0044] Through the above implementation process, the scene intelligent analysis model based on deep learning can efficiently parse the scene environment parameters into multiple structured functional layers, each layer contains complete geometric information, material information, interaction rules and environmental constraints, providing a solid foundation for subsequent Metaverse scene rendering and interaction. When processing complex urban scenes, the average processing time of this model does not exceed 5 seconds, the accuracy of functional layer division reaches 92%, and the constraint satisfaction reaches 95%, which greatly improves the efficiency and quality of Metaverse scene construction.

[0045] In an optional embodiment, A two-layer constraint optimization model is constructed, wherein the two-layer constraint optimization model includes local constraints within a functional layer and global constraints between functional layers, and constraint optimization is achieved by minimizing a weighted combination of constraint expected deviations and constraint differences between adjacent functional layers, including: Constructing local constraint features of the scene function layer, the local constraint features include geometric constraint features and material constraint features, wherein the geometric constraint features include position continuity constraints, normal vector consistency constraints and boundary continuity constraints, and the material constraint features include texture transition constraints, roughness continuity constraints and reflectivity consistency constraints; Constructing global constraint features of scene function layers, the global constraint features include inter-layer topological constraints and physical constraints, wherein the inter-layer topological constraints calculate the constraint relationship between adjacent functional layers through inter-layer association weights and topological relationship measurement functions, and the physical constraints include gravity constraints, collision detection constraints, and structural stability constraints; Constructing a multi-level optimization objective function based on the local constraint features and the global constraint features, taking a weighted combination of the geometric constraints and material constraints of the local constraint features as a local optimization objective, and taking a weighted combination of the topological constraints and physical constraints of the global constraint features as a global optimization objective; Adopting an adaptive weight adjustment mechanism to dynamically balance the local optimization target and the global optimization target, the adaptive weight is adjusted exponentially based on the ratio of the local optimization target value to the global optimization target value; Iteratively solving the multi-level optimization objective function by an alternating direction multiplier method until the difference between the optimization results of two adjacent iterations is less than a preset deviation threshold; The local constraint satisfaction and the global constraint satisfaction of each functional layer are calculated, and the local constraint satisfaction and the global constraint satisfaction are both calculated using a Gaussian evaluation function.

[0046] Exemplarily, local constraint features of the scene function layer are constructed, including geometric constraint features and material constraint features. The geometric constraint feature mainly focuses on the rationality of the internal geometric structure of the functional layer. The position continuity constraint ensures a smooth transition of the spatial positions of adjacent geometric elements in the same functional layer to avoid jump faults. For example, in the terrain function layer, the system samples the height values ​​of adjacent grid points. When the height difference of adjacent points exceeds the preset threshold (usually 25% of the grid spacing), the position continuity constraint is triggered and the transition is smoothed by interpolation. The normal vector consistency constraint ensures that the normal vector of the surface of the functional layer changes gradually and improves the rendering quality. During implementation, the system calculates the angle between the normal vectors of adjacent facets. When the angle exceeds 45 degrees, it is smoothed by subdivision surface technology to make the normal vector transition natural. The boundary continuity constraint processes the edge area of ​​the functional layer to prevent suspension or fracture. At the junction of the building function layer and the terrain function layer, the system detects the connection of the boundary points to ensure that the building base fits closely with the terrain surface, and the deviation is controlled within 2 cm.

[0047] The material constraint feature mainly deals with the consistency of material properties within the functional layer. The texture transition constraint achieves a natural transition between different materials through map fusion technology. In the area where the grass and sand meet, the system creates a 30-pixel-wide transition zone and applies an alpha blending algorithm to blend the two textures naturally. The roughness continuity constraint ensures the smooth change of the material roughness parameter and prevents abrupt material boundaries. The system divides the surface of the functional layer into multiple material areas, and the gradient change of the roughness parameter between areas is controlled within a range of no more than 0.2 per meter. The reflectivity consistency constraint ensures that similar material areas have consistent light reflection characteristics. For example, the reflectivity fluctuation within the same metal material area is controlled within ±5% to maintain visual consistency.

[0048] Next, construct the global constraint features of the scene function layer, including inter-layer topological constraints and physical constraints. The inter-layer topological constraints describe the dependence relationship between different function layers through the inter-layer association weight and the topological relationship metric function. The system establishes an association matrix for adjacent function layers. For example, the association weight between the terrain layer and the building layer is 0.85, indicating that the building layer highly depends on the terrain layer; while the association weight between the terrain layer and the special effect layer is only 0.3, indicating a lower degree of dependence. The topological relationship metric is comprehensively evaluated through three indicators: spatial overlap rate, geometric intersection, and functional dependence degree, generating a standardized topological relationship value within the range of 0-1.

[0049] The physical constraints mainly ensure the physical rationality of the scene. The gravity action constraint considers the stable state of objects in the scene under the action of gravity to prevent floating objects. When implemented, the system calculates the ratio of the support area to the mass for the objects in each function layer. When this ratio is lower than the safety threshold (0.2 square meters / ton), the constraint is triggered to adjust the position of the object. The collision detection constraint prevents objects between different function layers from penetrating each other. The system uses the spatial hashing algorithm to accelerate collision detection, divides the scene space into voxel grids of 1 meter × 1 meter × 1 meter, and only detects the collision possibility of objects within adjacent voxels, with the detection accuracy reaching the centimeter level. The structural stability constraint ensures the physical rationality of complex structures such as buildings and prevents the appearance of impossible suspended structures. The system evaluates the stability of the structure based on a simplified physical model, calculates the distribution of support points, the position of the center of gravity, and the force balance to ensure that the stability coefficient of the structure is greater than 1.5.

[0050] Based on the local constraint features and global constraint features, the system constructs a multi-level optimization objective function. The local optimization objective takes the weighted combination of geometric constraints and material constraints as the evaluation index, and the default weight assignment is 0.6 for geometric constraints and 0.4 for material constraints. For example, in the terrain function layer, the weight of the position continuity constraint is 0.5, the normal vector consistency constraint is 0.3, and the boundary continuity constraint is 0.2, which are comprehensively formed into a geometric constraint score; the weight of the texture transition constraint is 0.4, the roughness continuity constraint is 0.4, and the reflectivity consistency constraint is 0.2, which are comprehensively formed into a material constraint score.

[0051] The global optimization objective takes the weighted combination of topological constraints and physical constraints as the evaluation index, and the default weight is 0.5 for topological constraints and 0.5 for physical constraints. In the urban scene, the weight assignment of the inter-layer topological constraints is: 0.3 for the overlap degree index, 0.3 for geometric intersection, and 0.4 for functional dependence degree; the weight assignment of the physical constraints is: 0.3 for the gravity action constraint, 0.4 for the collision detection constraint, and 0.3 for the structural stability constraint.

[0052] The system uses an adaptive weight adjustment mechanism to dynamically balance local optimization goals and global optimization goals. The initial weights are set to 0.5 for local optimization and 0.5 for global optimization, and then dynamically adjusted according to the ratio of local target value to global target value during the optimization process. When the local optimization completion is significantly lower than the global optimization, the local weight will automatically increase, so that the local constraints get more optimization resources; and vice versa. For example, when the local optimization target value is 0.75 and the global optimization target value is 0.95, the local / global ratio is 0.789, and the system calculates the weight adjustment factor as 1.15, adjusts the local optimization weight to 0.575, and the global optimization weight to 0.425, so that more optimization resources are allocated to local constraints.

[0053] The optimization solution uses the alternating direction multiplier method, which decomposes the complex constraint optimization problem into multiple sub-problems and solves them alternately. The system first optimizes the local constraints and fixes the global constraints; then fixes the local constraints and optimizes the global constraints; finally, the overall solution is updated by combining the results of the two. This process is iterated repeatedly until the difference between the optimization results of two adjacent iterations is less than the preset threshold (usually set to 0.01). In urban scene optimization, the system generally requires 10-15 iterations to reach convergence conditions, and each iteration takes about 0.5 seconds.

[0054] After the optimization is completed, the system calculates the constraint satisfaction of each functional layer. The local constraint satisfaction is calculated by a Gaussian evaluation function, centered on the expected value of the constraint, and exponentially decays as the deviation increases. For example, the position continuity constraint satisfaction is 0.97, indicating that the deviation between the current solution and the expected constraint is very small; the global constraint satisfaction also uses a Gaussian evaluation function, but the evaluation range is extended to adjacent functional layers. For example, the collision detection constraint satisfaction is 0.92, indicating that the collision between functional layers in the current solution is within an acceptable range.

[0055] Through the above two-layer constraint optimization model, the system can effectively balance the internal consistency of the scene and the coordination between layers, generate physically reasonable and visually coherent metaverse scene environment constraints, and provide a solid foundation for subsequent scene rendering and interaction. In practical applications, this optimization method can increase the constraint satisfaction from the initial 72% to more than 95%, greatly improving the scene quality and user experience.

[0056] Table 1 is a table comparing the computational efficiency under different scene types. From the data, it can be seen that this technical solution has significant performance advantages in all scene types. In indoor scenes with low complexity (number of constraints: 245), this solution only takes 127.3ms, which is 34.3% more efficient than the single-layer constraint model (193.8ms) and the traditional method (298.7ms). As the complexity increases, the advantages of this solution are still obvious. In high-complexity indoor scenes (number of constraints: 1205), this solution takes 476.2ms, while the traditional method takes 1142.5ms.

[0057] Table 1: Comparison of computing efficiency under different scene types;

[0058] This application innovatively proposes a two-layer constraint structure, which divides constraints into two levels: local constraints within the functional layer and global constraints between functional layers, to systematically handle complex constraint relationships in the metaverse scene. Compared with the single-layer constraint model in the prior art, the two-layer constraint structure can more accurately describe the dependencies and interaction rules between different functional layers in the scene. For example, when dealing with the relationship between the building layer and the terrain layer, the traditional method only considers simple collision detection, while this application simultaneously considers multi-dimensional constraints such as position continuity, normal vector consistency, and material transition, making the scene more realistic and coordinated.

[0059] Secondly, this application introduces an adaptive weight adjustment mechanism to dynamically adjust and optimize resource allocation according to the completion of local constraints and global constraints during the optimization process. Compared with the fixed weight or simple linear adjustment in the prior art, this application adopts an exponential decay adjustment method based on a ratio, which can respond more quickly to changes in the constraint optimization state and avoid falling into a local optimal solution. Practical applications have shown that in the optimization of complex urban scenes, the adaptive weight mechanism reduces the number of iterations by an average of 30% compared to the fixed weight method, and the optimization efficiency is significantly improved.

[0060] This application adopts a refined constraint classification system, which systematically integrates geometric constraints, material constraints, topological constraints and physical constraints to form a comprehensive constraint evaluation system. Compared with the existing technology, the constraint granularity is finer, the coverage is wider, and the adaptability is stronger. Especially in terms of material constraints, three subdivision constraints of texture transition, roughness continuity and reflectivity consistency are introduced, filling the gap in the existing technology in material coherence processing.

[0061] In an optional embodiment, Calling the physical engine module, constructing a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and implanting corresponding physical attribute parameters in each three-dimensional mesh model includes: Constructing an adaptive tetrahedral mesh according to the geometric information, performing quality evaluation and mesh unit optimization on the tetrahedral mesh by mesh unit volume ratio, mesh unit dihedral angle and mesh unit side length ratio, and generating a high-quality tetrahedral mesh; Calculating physical property parameters based on the material information, respectively calculating mass density, elastic modulus, Poisson's ratio and friction coefficient of the scene function layer through a material-physical mapping network, and establishing a corresponding relationship between the material information and the physical property parameters; The physical property parameters are distributed to each mesh unit of the high-quality tetrahedral mesh to generate a three-dimensional physical mesh model with physical properties.

[0062] For example, in the process of quickly building a metaverse scene based on a modular engine, the call of the physical engine module is the key link to realize the real physical interaction of the scene. This module builds a three-dimensional mesh model based on the geometric information and material information of the scene function layer, and implants the corresponding physical property parameters to provide a physical simulation basis for the scene.

[0063] Exemplarily, an adaptive tetrahedral mesh is constructed according to the geometric information of the scene functional layer. The geometric surface of the functional layer is voxelized using the Delaunay tetrahedral partitioning algorithm, which takes the surface triangular mesh as input and generates a tetrahedral mesh structure that fills the interior. In specific implementation, the input surface triangular mesh is first converted into a signed distance field, with the mesh surface as the zero isosurface, the interior as positive values, and the exterior as negative values. Then, the density distribution function of the initial tetrahedral mesh is set, which takes into account the local geometric features of the model, such as curvature, thickness, and surface details, so that the mesh density is higher in areas with rich geometric features, and lower in flat areas. For example, for the column structure in the functional layer of the building, due to its load-bearing characteristics, a denser tetrahedral mesh is automatically generated inside the column, and the side length of the tetrahedral unit is controlled at about 5 cm; while for the plane part of the wall, the side length of the mesh unit can be relaxed to 20 cm to balance the simulation accuracy and computational efficiency.

[0064] After the tetrahedral mesh is generated, the mesh quality is evaluated by three key indicators: mesh unit volume ratio, mesh unit dihedral angle and mesh unit side length ratio. The mesh unit volume ratio refers to the ratio of the volume of a single tetrahedron to the volume of its circumscribed sphere, and the ideal value should be greater than 0.1; the mesh unit dihedral angle refers to the dihedral angle between adjacent faces of the tetrahedron, and too small (<10°) or too large (>170°) angles should be avoided; the mesh unit side length ratio refers to the ratio of the longest side to the shortest side, which should be controlled within 10. For mesh units that do not meet the quality requirements, local re-meshing technology is used for optimization, including edge reduction, edge flipping, node smoothing and unit splitting. For example, for a complex tree model in a game scene, about 12,000 tetrahedral units are generated in the main part of the trunk. In the initial mesh quality assessment, 27% of the units did not meet the standards. After 5 rounds of local optimization, the proportion of unqualified units dropped to 3%, meeting the requirements of physical simulation.

[0065] The physical property parameters are calculated based on the material information of the functional layer. The material-physical mapping network constructed by deep learning consists of a three-layer fully connected neural network. The input is the feature vector of the material (including color, roughness, metalness, normal map and other information), and the output is four key physical property parameters: mass density, elastic modulus, Poisson's ratio and friction coefficient. The hidden layer dimension of the network is [256, 128, 64], and the ReLU activation function is used. The network is trained with a large number of real material samples to establish a mapping relationship between the material appearance characteristics and physical properties. For example, for wooden materials, different types such as oak, pine, and walnut can be distinguished according to their texture characteristics and color depth, and the corresponding physical parameters are assigned: the mass density of oak is about 700 kg / m3, the elastic modulus is about 12 GPa, the Poisson's ratio is 0.3, and the static friction coefficient is 0.4; while metal materials have higher mass density (7800 kg / m3) and elastic modulus (200 GPa).

[0066] In order to improve the accuracy of material-physical mapping, a material mixing processing mechanism is also introduced to handle composite materials commonly seen in real scenes. When a surface is identified as having mixed features of multiple materials, the proportion of each component material is calculated, and the final physical properties are calculated using a weighted average. For example, for a metal surface covered with rust, the system may identify 70% metal features and 30% rust features, and the final assigned mass density is 0.7×7800+0.3×5200=7020 kg / m3. In addition, the anisotropy of the material is also considered, especially for materials such as wood and composite materials with obvious differences in physical properties in different directions. The elastic properties are represented by tensors to make the simulation results more realistic.

[0067] Finally, the calculated physical property parameters are assigned to each mesh unit of the high-quality tetrahedral mesh to generate a three-dimensional physical mesh model with physical properties. In the parameter assignment process, three different assignment strategies are supported: uniform assignment, gradient assignment, and hierarchical assignment. Uniform assignment is suitable for functional layers with uniform materials, and all mesh units are assigned the same physical parameters; gradient assignment is suitable for functional layers with gradient materials, and the physical parameters change gradiently along a specific direction; hierarchical assignment is suitable for multi-layer composite structures, and mesh units at different depths are assigned different physical parameters. For example, for the floor functional layer, it may be identified that the surface layer is a wood material, the middle layer is a sound insulation material, and the bottom layer is a cement structure. Based on this, a hierarchical assignment strategy is adopted to make the surface layer units have the physical properties of wood (low density, medium elasticity), the middle layer units have the properties of sound insulation materials (extremely low density, low elasticity), and the bottom layer units have the properties of cement (high density, high stiffness).

[0068] After the parameter assignment is completed, a spatial consistency check of the physical properties is performed to ensure that the physical properties between adjacent grid cells change smoothly and avoid numerical instability caused by mutations. For example, when it is detected that the elastic modulus difference between adjacent cells exceeds 50%, a transition unit is automatically created in the boundary area to make the physical properties transition smoothly. For the floor model of the city square scene (containing about 50,000 tetrahedral units), 5 main material areas were identified, and after assigning the corresponding physical parameters, about 2,000 transition units were created to ensure the stability of the physical simulation.

[0069] Table 2 is a comparative experimental table of mesh quality evaluation. This technology has achieved significant improvements in the minimum dihedral angle, from 12.4 degrees of the traditional method to 23.8 degrees, with an improvement rate of 91.9%, which means that the mesh quality has been greatly improved. At the same time, the maximum dihedral angle has also been optimized, from 167.2 degrees to 156.3 degrees, with an improvement rate of 6.5%. The average unit volume ratio has increased from 0.683 to 0.872, an improvement of 27.7%, indicating that the mesh generated by this technology is more uniform. In terms of the mesh unit side length ratio, this technology has achieved an improvement from 2.42 to 1.35, with an improvement rate of 44.2%, close to the ideal value of 1.0. The Hausdorff distance error has been greatly reduced, from 1.83mm to 0.57mm, with an improvement rate of 68.9%, indicating that the geometric approximation accuracy is higher. The volume retention rate has increased from 96.2% to 99.8%, ensuring the accuracy of the model volume. In addition, the number of mesh elements required by this technology was reduced by 23.7%, from 24568 to 18734, which not only improved the computational efficiency but also ensured the mesh quality.

[0070] Table 2: Mesh quality evaluation comparison experiment table;

[0071] The three-dimensional physical mesh model generated by the above method not only retains the precise expression of the geometric appearance, but also accurately maps the physical properties corresponding to the material, providing a solid foundation for subsequent physical interaction simulation. In actual tests, the physical mesh model generated by this method can support real-time physical interaction at up to 100 frames per second, while maintaining the authenticity of physical behavior and meeting the high requirements of the metaverse scene for physical interaction. For example, when simulating a sphere hitting surfaces of different materials, it can accurately reproduce the rebound height, sound characteristics, and deformation behavior of different materials, greatly enhancing the user's sense of immersion and interactive experience.

[0072] In an optional embodiment, Calling the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layer includes: The hierarchical relationship includes the spatial position relationship and occlusion relationship between the functional layers; Based on the hierarchical relationship, a hierarchical illumination propagation tree is constructed, wherein the nodes of the hierarchical illumination propagation tree represent scene function layers, the connecting edges between the nodes represent illumination propagation paths, and an illumination attenuation weight is assigned to each illumination propagation path according to the spatial position relationship and the occlusion relationship; According to the structure of the hierarchical illumination propagation tree, direct illumination and indirect illumination of each of the three-dimensional grid models are calculated to generate an illumination interaction matrix, wherein the illumination interaction matrix records the light energy transfer relationship between different functional layers; A depth map of each of the three-dimensional mesh models is generated based on the illumination interaction matrix, and a shadow mapping matrix is ​​calculated in combination with the illumination attenuation weights, and the illumination distribution of each functional layer is dynamically updated and optimized according to the shadow mapping matrix.

[0073] Exemplarily, the hierarchical relationship between the functional layers of the scene is clarified, including the spatial position relationship and the occlusion relationship. The spatial position relationship is calculated through the spatial coordinates and bounding volume of each functional layer. The system creates an axial bounding box for each functional layer, recording its center coordinates, length, width, height and rotation angle in the world space. For example, in the city square scene, the bounding box size of the ground functional layer is 500×500×2 meters, and the center coordinates are (0,0,0); the bounding box size of the building functional layer is 80×50×30 meters, and the center coordinates are (60,40,15). The occlusion relationship is determined by the ray casting algorithm. The system projects light from the main light source position to the sampling points of each functional layer, and determines the occlusion relationship by detecting the intersection of light with other functional layers. In actual implementation, the system uses hierarchical space division technology (octree) to accelerate ray casting calculations, recursively divide the space into sub-areas, and the light only needs to be tested for intersection with the areas that may intersect, reducing the computational complexity from O(n) to O(log n), where n is the number of functional layers in the scene.

[0074] After obtaining the hierarchical relationship, the system constructs a hierarchical lighting propagation tree. Each node of the tree structure represents a scene functional layer, and the connecting edges between the nodes represent the propagation path of the lighting energy. The root node is usually set as the main light source (such as sunlight or main artificial light source), and then the branch structure of the tree is constructed according to the spatial relationship and visibility of the functional layers. For example, for the functional layer directly illuminated by the main light source (such as the open ground and the exterior wall of the building), it is used as the direct child node of the root node; and the area blocked by other functional layers (such as the ground part under the shadow of the building) is used as the child node of the corresponding occlusion layer node. For each lighting propagation path (that is, the edge in the tree), the system assigns a lighting attenuation weight, which comprehensively considers the following factors: distance attenuation, angle attenuation, degree of occlusion, and material reflection characteristics.

[0075] Distance attenuation is calculated based on the inverse square relationship between the light source and the receiving surface. For example, when the distance increases from 5 meters to 10 meters, the light intensity drops to 1 / 4 of the original. Angle attenuation takes into account the angle between the incident angle of the light and the surface normal. When the incident angle increases from 0 degrees (vertical incidence) to 60 degrees, the effective light intensity drops to 1 / 2 of the original. The degree of occlusion is determined by calculating the proportion of the overlapping projection areas between the functional layers. For example, when building A blocks 30% of the area receiving light from building B, the weight of the corresponding light path will be multiplied by an attenuation coefficient of 0.7. The material reflection characteristics are based on the reflectivity and scattering characteristics of the surface material. For example, the reflectivity of high-gloss metal materials can reach more than 0.8, while the reflectivity of rough concrete surfaces is only about 0.2.

[0076] In the city square scene example, the weight of the direct lighting path from the main light source (sun) to the ground functional layer is 0.9 (considering the slight attenuation of atmospheric scattering); the weight of the indirect lighting path from the ground to the building exterior wall is 0.15 (considering the combined effect of ground reflectivity and distance attenuation); the weight of the indirect lighting path from the building exterior wall to the adjacent building is 0.08 (considering the combined effect of wall reflectivity and angle attenuation). These weight values ​​collectively constitute the edge weight network of the light propagation tree, which accurately describes the transmission law of light energy in the scene.

[0077] Based on the constructed hierarchical lighting propagation tree, the system calculates the direct and indirect lighting of each three-dimensional mesh model. The direct lighting calculation adopts the Phong lighting model, taking into account the three components of ambient light, diffuse light, and specular light. For example, for the exterior wall of the building, the system sets the ambient light coefficient to 0.2 (indicating that there is still 20% basic brightness in the case of complete occlusion), the diffuse reflection coefficient to 0.6 (indicating the surface diffuse reflection ability), and the specular reflection coefficient to 0.1 (indicating the ability of the surface to produce highlights). The indirect lighting calculation is based on the multi-level jump path in the lighting propagation tree, and accumulates the light energy that reaches the mesh surface after multiple reflections. For example, for the interior corridor of the building, its lighting comes not only from the sunlight directly through the window (direct lighting), but also from the light reflected from the exterior wall, the ground, and the adjacent buildings (indirect lighting).

[0078] The system organizes the calculation results of direct and indirect lighting into a lighting interaction matrix, which is an n×n square matrix (n is the number of functional layers). The matrix element M[i,j] represents the proportion of light energy transferred from functional layer i to functional layer j. In the urban square scene with 10 functional layers, the generated lighting interaction matrix contains 100 elements, which accurately records the light energy transfer relationship between each functional layer. For example, M[2,5]=0.35 means that the proportion of light energy reflected from the building exterior wall (functional layer 2) to the decorative plants (functional layer 5) is 35%.

[0079] Based on the lighting interaction matrix, the system generates a depth map of the scene for each light source perspective. A depth map is a special image whose pixel values ​​represent the distance from the light source to each point in the scene. The system uses a shadow mapping algorithm. First, the light source is set as the rendering perspective and the scene is rendered in grayscale. The grayscale value is proportional to the depth. For complex scenes, the system uses cascaded shadow mapping technology to divide the light source cone into three areas: near, medium, and far. Depth maps of different resolutions are generated respectively. A 512×512 high-resolution depth map is used for the near area, and a 128×128 low-resolution depth map is used for the far area to balance accuracy and efficiency.

[0080] After obtaining the depth map, the system calculates the shadow mapping matrix in combination with the lighting attenuation weights. The matrix records the shadow attenuation coefficient for each sampling point in the scene, ranging from 0 (completely in shadow) to 1 (completely illuminated). During calculation, the system renders the scene from the current perspective, and for each visible fragment, projects it back to the light source space, compares the depth value of the fragment in the light source space with the value recorded in the depth map, and if the former is greater than the latter (plus a small offset to avoid self-occlusion), the fragment is in the shadow. In order to avoid jagged hard shadow edges, the system uses percentage proximity filtering technology to sample 9 points in a 3×3 area around the fragment, and calculates the proportion of sampling points in the shadow as the shadow attenuation coefficient to generate soft and natural shadow edges.

[0081] The lighting distribution of each functional layer is dynamically updated and optimized according to the shadow mapping matrix. Specifically, the system multiplies the initial calculated lighting result with the shadow mapping matrix to obtain the final lighting distribution that takes into account the shadow effect. For example, if the initial lighting intensity of a point is 0.8 and the corresponding attenuation coefficient in the shadow mapping matrix is ​​0.6, the final lighting intensity of the point is 0.48. In order to further enhance the visual effect, the system implements adaptive ambient occlusion technology to appropriately increase the ambient light intensity in the shadow area to avoid the shadow area being too dark and improve the visibility of details. In indoor scenes, the ambient light intensity in the shadow area will increase by about 30%, so that the details in the shadow area are still visible.

[0082] Through the above implementation method, the system can generate highly realistic lighting and shadow effects for the metaverse scene. In actual tests, the scene processed by this lighting rendering module has a lighting calculation speed of about 40% compared to the traditional method, and the shadow quality score (taking into account the softness of the shadow edge, detail retention and visual coherence) has been improved from 7.2 to 9.1 (out of 10 points), which significantly enhances the immersion and realism of the scene.

[0083] A second aspect of an embodiment of the present invention provides a metaverse scene rapid construction system based on a modular engine, including: The first unit is used to obtain a user's creation instruction for a metaverse scene, wherein the creation instruction includes scene type information and scene environment parameters; The second unit is used to call a corresponding scene rendering module from a preset modular engine library according to the scene type information, wherein the scene rendering module includes a physical engine module, a lighting rendering module and an environmental special effect module; The third unit is used for extracting features and stratifying the scene environment parameters based on a deep learning scene intelligent analysis model, parsing the scene environment parameters into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints; a progressive hierarchical relationship is formed between the scene function layers, and the environmental constraints of the upper scene function layer will optimize and adjust the rendering results of the lower scene function layer; The fourth unit is used to call the physical engine module, construct a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and implant corresponding physical property parameters in each three-dimensional mesh model; A fifth unit is used to call the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layers; The sixth unit is used to call the environmental special effects module and add corresponding environmental special effects to each of the scene function layers according to the scene environment parameters.

[0084] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0085] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0086] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for quickly building a metaverse scene based on a modular engine, characterized in that: include: Obtaining a user's creation instruction for a metaverse scene, wherein the creation instruction includes scene type information and scene environment parameters; According to the scene type information, a corresponding scene rendering module is called from a preset modular engine library, wherein the scene rendering module includes a physical engine module, a lighting rendering module, and an environmental special effect module; Based on the deep learning scene intelligent analysis model, the scene environment parameters are feature extracted and the scene is layered, and the scene environment parameters are parsed into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints; a progressive hierarchical relationship is formed between the scene function layers, and the environmental constraints of the upper scene function layer will optimize and adjust the rendering results of the lower scene function layer; Calling the physical engine module to construct a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and embedding corresponding physical property parameters in each three-dimensional mesh model; Calling the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layers; The environmental special effects module is called to add corresponding environmental special effects to each of the scene function layers according to the scene environment parameters.

2. The method according to claim 1, characterized in that According to the scene type information, calling a corresponding scene rendering module from a preset modular engine library includes: Receiving scene type information, and encoding the scene type information through a pre-trained BERT model to obtain a scene feature vector; A candidate rendering module is obtained from the modular engine library, and the matching degree between the candidate rendering module and the scene feature vector is calculated. The matching degree is calculated as follows: ； in, M Represents a set of candidate rendering modules, S represents the scene feature vector set, α represents the balance factor, n represents the feature dimension, w i represents the weight coefficient of the i-th dimension feature, sim(m i ,s i ) represents the similarity between the candidate rendering module of the i-th dimension and the feature vector of the i-th dimension scene, β represents the balance factor, C(M) Represents the computational complexity evaluation value corresponding to the candidate rendering module set, C () represents the complexity evaluation function; The candidate rendering module with the highest matching degree is selected as the scene rendering module corresponding to the scene type information.

3. The method according to claim 1, characterized in that Based on the deep learning scene intelligent analysis model, the scene environment parameters are feature extracted and the scene is layered, and the scene environment parameters are parsed into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints, including: Extracting geometric features and material features from the scene environment parameters respectively through a multimodal feature extraction network, wherein the geometric features are extracted from a scene point cloud set through a PointNet++ network, and the material features are extracted from a material map through a multi-scale convolutional neural network; A multi-head attention mechanism is used to fuse the geometric features and the material features, the geometric features are used as query vectors, the material features are used as key vectors, and the concatenation result of the geometric features and the material features is used as a value vector to generate fused features; Performing hierarchical analysis on the fused features, dividing the scene into multiple functional layers, and establishing a hierarchical relationship between the functional layers; Generate geometric information, material information and interaction rules of each functional layer through a multi-layer perceptron respectively; A two-layer constraint optimization model is constructed, which includes local constraints within a functional layer and global constraints between functional layers, and constraint optimization is achieved by minimizing a weighted combination of constraint expected deviations and constraint differences between adjacent functional layers.

4. The method according to claim 3, characterized in that A two-layer constraint optimization model is constructed, wherein the two-layer constraint optimization model includes local constraints within a functional layer and global constraints between functional layers, and constraint optimization is achieved by minimizing a weighted combination of constraint expected deviations and constraint differences between adjacent functional layers, including: Constructing local constraint features of the scene function layer, the local constraint features include geometric constraint features and material constraint features, wherein the geometric constraint features include position continuity constraints, normal vector consistency constraints and boundary continuity constraints, and the material constraint features include texture transition constraints, roughness continuity constraints and reflectivity consistency constraints; Constructing global constraint features of scene function layers, the global constraint features include inter-layer topological constraints and physical constraints, wherein the inter-layer topological constraints calculate the constraint relationship between adjacent functional layers through inter-layer association weights and topological relationship measurement functions, and the physical constraints include gravity constraints, collision detection constraints, and structural stability constraints; Constructing a multi-level optimization objective function based on the local constraint features and the global constraint features, taking a weighted combination of the geometric constraints and material constraints of the local constraint features as a local optimization objective, and taking a weighted combination of the topological constraints and physical constraints of the global constraint features as a global optimization objective; Adopting an adaptive weight adjustment mechanism to dynamically balance the local optimization target and the global optimization target, the adaptive weight is adjusted exponentially based on the ratio of the local optimization target value to the global optimization target value; Iteratively solving the multi-level optimization objective function by an alternating direction multiplier method until the difference between the optimization results of two adjacent iterations is less than a preset deviation threshold; The local constraint satisfaction and the global constraint satisfaction of each functional layer are calculated, and the local constraint satisfaction and the global constraint satisfaction are both calculated using a Gaussian evaluation function.

5. The method according to claim 1, characterized in that Calling the physical engine module, constructing a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and implanting corresponding physical attribute parameters in each three-dimensional mesh model includes: Constructing an adaptive tetrahedral mesh according to the geometric information, performing quality evaluation and mesh unit optimization on the tetrahedral mesh by mesh unit volume ratio, mesh unit dihedral angle and mesh unit side length ratio, and generating a high-quality tetrahedral mesh; Calculating physical property parameters based on the material information, respectively calculating mass density, elastic modulus, Poisson's ratio and friction coefficient of the scene function layer through a material-physical mapping network, and establishing a corresponding relationship between the material information and the physical property parameters; The physical property parameters are distributed to each mesh unit of the high-quality tetrahedral mesh to generate a three-dimensional physical mesh model with physical properties.

6. The method according to claim 1, characterized in that Calling the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layer includes: The hierarchical relationship includes the spatial position relationship and occlusion relationship between the functional layers; Based on the hierarchical relationship, a hierarchical illumination propagation tree is constructed, wherein the nodes of the hierarchical illumination propagation tree represent scene function layers, the connecting edges between the nodes represent illumination propagation paths, and an illumination attenuation weight is assigned to each illumination propagation path according to the spatial position relationship and the occlusion relationship; According to the structure of the hierarchical illumination propagation tree, direct illumination and indirect illumination of each of the three-dimensional grid models are calculated to generate an illumination interaction matrix, wherein the illumination interaction matrix records the light energy transfer relationship between different functional layers; A depth map of each of the three-dimensional mesh models is generated based on the illumination interaction matrix, and a shadow mapping matrix is ​​calculated in combination with the illumination attenuation weights, and the illumination distribution of each functional layer is dynamically updated and optimized according to the shadow mapping matrix.

7. A modular engine-based metaverse scene rapid construction system, used to implement the method as described in any one of claims 1 to 6, characterized in that: include: The first unit is used to obtain a user's creation instruction for a metaverse scene, wherein the creation instruction includes scene type information and scene environment parameters; The second unit is used to call a corresponding scene rendering module from a preset modular engine library according to the scene type information, wherein the scene rendering module includes a physical engine module, a lighting rendering module and an environmental special effect module; The third unit is used for extracting features and stratifying the scene environment parameters based on a deep learning scene intelligent analysis model, parsing the scene environment parameters into multiple scene function layers, each of which includes corresponding geometric information, material information, interaction rules and environmental constraints; a progressive hierarchical relationship is formed between the scene function layers, and the environmental constraints of the upper scene function layer will optimize and adjust the rendering results of the lower scene function layer; The fourth unit is used to call the physical engine module, construct a three-dimensional mesh model according to the geometric information and material information of each scene function layer, and implant corresponding physical property parameters in each three-dimensional mesh model; A fifth unit is used to call the lighting rendering module to perform lighting calculation and shadow mapping on each of the three-dimensional mesh models based on the hierarchical relationship of the scene function layers; The sixth unit is used to call the environmental special effects module and add corresponding environmental special effects to each of the scene function layers according to the scene environment parameters.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images