A 3D VR engine scene automatic construction method based on scene semantic recognition

Abstract

The application discloses a kind of 3DVR engine scene automatic construction methods based on scene semantic identification, and the method specifically includes: obtaining multi-modal environment data, constructs point cloud map and executes deep neural network semantic segmentation, calculates target object space posture, constructs topological scene space relation graph, according to semantic node searches three-dimensional model asset and self-adapting adjustment grid coordinate, imports target model into rendering engine, calculates instance transformation matrix to complete automation building, and executes collision detection and material rendering.Solve the problem that traditional manual construction scene is inefficient and unstructured data is difficult to directly participate in the complex logic interaction of domestic light three-dimensional virtual reality engine, with high degree of automation, bottom logic is rigorous, and can provide high-fidelity scene interaction and efficient asset management for the technical advantages of high-end equipment manufacturing-oriented industrial intelligent platform.

Description

Technical Field

[0001] This application relates to the fields of 3D scene and computer vision technology, and in particular to a method for automatic scene construction in a 3DVR engine based on scene semantic recognition. Background Technology

[0002] With the rapid development of spatial computing and related concepts of the metaverse, the way humans interact with the digital world is undergoing a profound transformation from two-dimensional screens to three-dimensional immersive spaces. Cutting-edge technologies such as virtual reality, augmented reality, and digital twins have widely penetrated multiple core fields, including industrial manufacturing, urban planning, medical simulation, and entertainment. Against this backdrop, constructing high-fidelity 3D virtual scenes with complex interactive features has become a core infrastructure supporting the implementation of these applications. However, with the continuous expansion of application dimensions, the market demand for high-quality 3D digital content is growing exponentially, and traditional content production models are facing unprecedented pressure and capacity challenges.

[0003] For a long time, the construction of 3D scenes has heavily relied on manual operations by professional artists or technical experts. Technicians need to use complex 3D software, going through tedious and highly homogenized processes such as original artwork design, topological modeling, material rendering, and lighting rendering. This labor-intensive production model is not only time-consuming and costly, but also fails to meet the current demand for rapid iteration of massive amounts of 3D scenes. On the other hand, although reverse engineering technologies such as 3D laser scanning and oblique photography can quickly acquire 3D point cloud data from the real world, this raw data often exhibits disordered and unstructured characteristics. It only contains spatial geometry and color information, lacking a deep understanding of the physical properties and logical relationships of scene entities. This superficial data reproduction, which "knows what but not why," prevents the generated mesh models from directly participating in the complex logical interactions and physical collision calculations within the virtual engine, greatly limiting their practical value in deeply interactive scenes.

[0004] To overcome the bottlenecks in 3D content production, the deep integration of artificial intelligence technologies, represented by deep learning, with 3D computer vision offers a novel solution. In particular, breakthroughs in key technologies such as 3D point cloud semantic segmentation, object detection, and multimodal feature extraction in recent years have enabled computers not only to accurately reconstruct the geometry of the 3D world but also to "understand" the semantic categories and inherent physical relationships of various objects within a scene. By transforming chaotic environmental perception data into structured information with rich labels, the system can accurately extract independent instances such as building structures, interior furniture, and industrial equipment, and further analyze their topological structures and constraints in space. This leap from "geometric perception" to "semantic cognition" lays a solid theoretical and data foundation for the automated and intelligent construction of 3D scenes.

[0005] The automated construction method based on scene semantic recognition is a natural product of this technological evolution. This technology aims to streamline the end-to-end data processing and content generation pipeline, automatically completing the entire process from real-world environment scanning, semantic feature extraction, standard 3D asset matching to automated assembly within the virtual engine through intelligent algorithms. It efficiently handles the massive and tedious manual construction work with underlying algorithms, significantly reducing the production cycle and marginal cost of 3D digital content. More importantly, scenes generated based on semantics inherently possess the advantages of modularity and object-orientation, with each entity retaining independent physical attributes and interaction interfaces. This provides great flexibility for subsequent secondary development, dynamic simulation, and cross-platform reuse. In conclusion, this technological path not only represents the core development direction of the next-generation 3D content generation paradigm but is also a key engine driving the virtual reality industry towards large-scale and widespread application. Summary of the Invention

[0006] This invention addresses the computational bottlenecks and excessive memory consumption faced by current 3D virtual reality engines when processing massive amounts of high-precision 3D models, as well as the topological destruction and loss of key geometric features easily caused by traditional mesh simplification algorithms. It proposes a lightweight 3D model processing method based on feature decoupling and dynamic recombination. This method decouples the geometry and surface details of the 3D model using high-dimensional features, separating the core skeleton topology and high-frequency geometric features. At the rendering end, it adaptively and dynamically re-maps the compressed features based on viewpoint distance and device computing power. This solves the technical challenge of balancing high-fidelity visual quality and high-frame-rate real-time interaction in cross-platform, multi-terminal deployments, providing core algorithmic support for the efficient operation of domestically developed lightweight 3DVR engines.

[0007] The objective of this invention is achieved through the following technical solutions. First, a high-precision 3D mesh model is acquired, and a graph neural network is used to extract the spatial geometric features of the model vertices and the local neighborhood topological features to complete the graph convolutional encoding operation of the low-level features. Second, a high-dimensional feature decoupling network is constructed to accurately separate the encoded mixed features into low-frequency basic skeleton features and high-frequency surface detail features. Simultaneously, implicit neural representation technology is used to perform feature vectorization compression on the high-frequency detail information to significantly reduce the model's storage volume. Next, an error-metric-driven edge-folding algorithm is used to generate a lightweight basic mesh with multiple resolution levels for the low-frequency basic skeleton features, ensuring that the mesh simplification process does not destroy the global topological coherence and physical boundaries of the target object. Finally, a viewpoint-aware dynamic reconstruction module is introduced into the real-time rendering pipeline. Based on the current spatial coordinates of the virtual camera and the remaining hardware computing resources, the required high-frequency feature detail levels are adaptively calculated and fused with the lightweight basic mesh in real time for reconstruction. This significantly reduces the rendering load on the graphics processor while achieving lossless visual representation of the 3D scene. Specifically, the following steps are included:

[0008] 1. A method for automatic scene construction in a 3DVR engine based on scene semantic recognition, the method comprising the following steps:

[0009] Step 1: Acquire multimodal environment perception data of the target scene and perform preprocessing operations to construct an initial 3D point cloud map. Then, use a deep neural network to perform point-level semantic segmentation on the point cloud map to obtain scene semantic information with high-precision category labels and confidence features.

[0010] 1.1 Collect environmental multimodal perception data containing depth information and color images, and use bilateral filtering algorithm to eliminate noise interference generated during data acquisition to improve the overall signal-to-noise ratio and quality of the original environmental data.

[0011] 1.2 Using the camera intrinsic and extrinsic parameter matrix, the pixel coordinates of the two-dimensional color image are mapped to the three-dimensional space to establish the mapping relationship between color information and spatial geometric structure, and complete the initial three-dimensional point cloud map stitching and registration work from multiple perspectives.

[0012] 1.3 Input the stitched 3D point cloud map into the pre-trained 3D semantic segmentation model to extract local geometric features and global topological structure, and output the semantic category prediction results and corresponding probability distribution of each 3D point.

[0013] Step 2: Calculate the geometric boundaries of the target objects based on the extracted scene semantic information and analyze the relative positions and orientations of different objects to construct a scene spatial relationship graph with topological connection attributes, thereby representing the physical dependencies and associations of various entity components in the 3D scene.

[0014] 2.1 Based on the same category labels in the semantic segmentation results, perform region growing and clustering segmentation on adjacent point cloud clusters to obtain independent instance objects, and use principal component analysis to solve for the minimum bounding box of each instance object.

[0015] 2.2 Extract the center three-dimensional coordinates and three-axis direction vectors of the bounding box of each instance object to calculate the Euclidean distance and relative deflection angle between each pair of objects, thereby quantifying the position and orientation constraints between spatial entities.

[0016] 2.3 Generate a scene spatial relationship graph by treating instance objects as nodes and relative position and orientation constraints as directed edges in the graph structure, and solidify the semantic category and bounding box size attributes of the nodes into the spatial relationship graph.

[0017] Step 3: Analyze the semantic features of the nodes in the scene spatial relationship graph and connect them to the pre-built 3D model asset library. Use a matching algorithm to retrieve candidate models that match the semantic tags and dimensions. At the same time, adaptively adjust the mesh vertex coordinates and scaling ratio of the candidate models based on the bounding box parameters.

[0018] 3.1 Extract semantic labels from nodes in the spatial relationship graph and convert them into standard retrieval descriptors. Then, perform hash feature indexing in the structured 3D model database to obtain an initial set of candidate 3D models with the same semantic attributes.

[0019] 3.2 Calculate the cosine similarity between the morphological descriptor of the candidate 3D model and the geometric features of the original point cloud instance object, sort them in descending order of similarity score, and then select the optimal matching target 3D model with the highest morphological realism.

[0020] 3.3 Compare the original size of the target 3D model with the physical size of the bounding box of the point cloud instance object, calculate the non-uniform scaling matrix in the 3D coordinate system, and apply the matrix to the target 3D model to complete the size normalization.

[0021] Step 4: Import the resized target 3D model into the virtual reality rendering engine. Combine the edge constraint information of the scene spatial relationship graph to calculate the instantiation transformation matrix of each model in the engine coordinate system, thereby completing the automated construction of the 3D scene physical layout and the binding of hierarchical node logical relationships.

[0022] 4.1 The normalized target 3D model is loaded into the virtual reality engine memory through a cross-platform interface, and a unique identifier is assigned to each model instance to facilitate subsequent rendering calls and hierarchical attribute control.

[0023] 4.2 Based on the relative deflection angle and translation distance defined in the scene space relationship diagram, construct quaternions and displacement vectors to solve the absolute transformation matrix of each model instance in the engine world coordinate system and update its space transformation components.

[0024] 4.3 Based on the inclusion and dependency relationships between target objects, establish a parent-child node hierarchy in the scene tree of the virtual reality engine to ensure that the linkage displacement and rotation operations of complex composite objects conform to the physical constraints of the real world.

[0025] Step 5: Perform physical collision detection and clipping repair on the initially constructed 3D virtual scene. At the same time, configure the lighting baking parameters and material reflection properties of the scene environment based on global semantic information. Finally, output a standardized virtual reality scene resource file that meets the needs of immersive interaction.

[0026] 5.1 Attach physical collider components to the entity models in the scene tree and use a ray detection algorithm to scan the overlapping areas of the models. For entities that have mesh penetration, perform a spatial backoff translation algorithm to eliminate unreasonable physical collision interference.

[0027] 5.2 Automatically load skyboxes and global illumination presets based on scene context. Utilize a physically based rendering pipeline to give the model surface the correct diffuse and specular material properties, thereby enhancing the visual expressiveness of virtual reality scenes.

[0028] 5.3 Summarize all mesh assets and material lighting data within the scene, perform serialization and packaging of rendering resources, and finally generate an independent executable program or digital asset package that can be directly read and run by virtual reality terminal devices.

[0029] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress.

[0030] 1) This invention physically separates the underlying geometric skeleton and surface details of a 3D model by decoupling high-dimensional features and combines implicit neural representation compression technology to significantly reduce the storage redundancy and video memory usage of model assets. This significantly alleviates the computing power bottleneck when the lightweight 3D engine processes large-scale scenes, thereby effectively improving the frame rate and smoothness of cross-terminal device rendering.

[0031] 2) The present invention introduces a viewpoint-aware dynamic reorganization mechanism that can adaptively map high-frequency detail features according to virtual space coordinates in the real-time rendering pipeline. While performing depth mesh simplification, it perfectly preserves the high-fidelity geometry of the entity and completely overcomes the defect of traditional algorithms that are prone to topological damage. It provides reliable technical support for high-precision demand scenarios such as high-end equipment manufacturing and industrial intelligent simulation. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the automatic scene construction method for a 3DVR engine based on scene semantic recognition, as proposed in this application.

[0033] Figure 2 is a schematic diagram of the specific operation of Example 1. Detailed Implementation

[0034] The specific implementation of this invention is as follows: First, high-precision 3D mesh model data of the target is acquired and its vertex coordinates and facet index information are parsed and transformed into an undirected attribute graph structure. The nodes of the graph structure are defined as the mesh vertices of the 3D model, and the edges are defined as the topological connections between vertices. A multi-layer graph convolutional neural network is constructed, and the spatial 3D coordinates and normal vectors of all nodes are used as initial input features to perform continuous graph convolution operations to extract a hybrid high-dimensional feature matrix containing local geometric curvature and global topological associations. Then, the extracted hybrid high-dimensional feature matrix is ​​input into a decoupled neural network based on a dual-branch architecture for feature separation. The first branch network outputs a low-frequency skeleton feature set representing the basic spatial physical architecture of the model through low-pass filtering aggregation. The second branch network outputs a high-frequency detail feature set representing the micro-texture undulations of the model surface through high-pass filtering. A multi-layer perceptron model is constructed, and the high-frequency detail feature set is used as the training target. Backpropagation algorithm is executed to fit it into a continuous implicit neural representation function. The weight parameters after the network convergence are extracted to generate implicit encoding compression vectors of high-frequency features. Subsequently, based on the low-frequency skeleton feature set, the geometric weights of the mesh vertices are requantized, and the quadratic error metric cost matrix of all edges of the target mesh is calculated. The mesh edges are globally sorted in ascending order of error cost to construct an edge folding priority queue. A vertex retention rate threshold for the target mesh is set, and edges with the lowest error cost are sequentially extracted from the priority queue to perform edge folding and topology shrinkage operations. The 3D coordinates of the new vertices and the adjacent edge structure after folding are dynamically updated until the set threshold is met, thereby generating a lightweight base mesh model with multi-level resolution and saving its hierarchical structure data. Finally, the lightweight base mesh model and the implicitly encoded compressed vectors of high-frequency features are synchronously loaded into the memory of the 3D virtual reality engine. During the engine's real-time rendering pipeline, the world space coordinates of the current virtual camera and the frustum parameters are obtained, and the spatial straight-line distance and observation deflection angle from the viewpoint to each mesh cluster of the lightweight base mesh model are calculated in real time. The viewpoint distance, observation angle, and implicitly encoded compressed vectors are input into the inference module of the engine's rendering thread for real-time decoding to generate surface elevation displacement data and normal perturbation maps corresponding to the viewpoint level. The computational shader using the graphical application programming interface (API) overlays the decoded displacement data and normal perturbation data vertex by vertex onto the corresponding lightweight base mesh, performing dynamic reconstruction of the 3D model geometry and the final rasterization rendering output.

[0035] See appendix Figure 1 The present invention provides a method for automatic scene construction in a 3DVR engine based on scene semantic recognition, which specifically includes the following steps:

[0036] 1. A method for automatic scene construction in a 3DVR engine based on scene semantic recognition, the method comprising the following steps:

[0037] Step 1: Acquire multimodal environment perception data of the target scene and perform preprocessing operations to construct an initial 3D point cloud map. Then, use a deep neural network to perform point-level semantic segmentation on the point cloud map to obtain scene semantic information with high-precision category labels and confidence features.

[0038] 1.1 Collect environmental multimodal perception data containing depth information and color images, and use bilateral filtering algorithm to eliminate noise interference generated during data acquisition to improve the overall signal-to-noise ratio and quality of the original environmental data.

[0039] 1.2 Using the camera intrinsic and extrinsic parameter matrix, the pixel coordinates of the two-dimensional color image are mapped to the three-dimensional space to establish the mapping relationship between color information and spatial geometric structure, and complete the initial three-dimensional point cloud map stitching and registration work from multiple perspectives.

[0040] 1.3 Input the stitched 3D point cloud map into the pre-trained 3D semantic segmentation model to extract local geometric features and global topological structure, and output the semantic category prediction results and corresponding probability distribution of each 3D point.

[0041] Step 2: Calculate the geometric boundaries of the target objects based on the extracted scene semantic information and analyze the relative positions and orientations of different objects to construct a scene spatial relationship graph with topological connection attributes, thereby representing the physical dependencies and associations of various entity components in the 3D scene.

[0042] 2.1 Based on the same category labels in the semantic segmentation results, perform region growing and clustering segmentation on adjacent point cloud clusters to obtain independent instance objects, and use principal component analysis to solve for the minimum bounding box of each instance object.

[0043] 2.2 Extract the center three-dimensional coordinates and three-axis direction vectors of the bounding box of each instance object to calculate the Euclidean distance and relative deflection angle between each pair of objects, thereby quantifying the position and orientation constraints between spatial entities.

[0044] 2.3 Generate a scene spatial relationship graph by treating instance objects as nodes and relative position and orientation constraints as directed edges in the graph structure, and solidify the semantic category and bounding box size attributes of the nodes into the spatial relationship graph.

[0045] Step 3: Analyze the semantic features of the nodes in the scene spatial relationship graph and connect them to the pre-built 3D model asset library. Use a matching algorithm to retrieve candidate models that match the semantic tags and dimensions. At the same time, adaptively adjust the mesh vertex coordinates and scaling ratio of the candidate models based on the bounding box parameters.

[0046] 3.1 Extract semantic labels from nodes in the spatial relationship graph and convert them into standard retrieval descriptors. Then, perform hash feature indexing in the structured 3D model database to obtain an initial set of candidate 3D models with the same semantic attributes.

[0047] 3.2 Calculate the cosine similarity between the morphological descriptor of the candidate 3D model and the geometric features of the original point cloud instance object, sort them in descending order of similarity score, and then select the optimal matching target 3D model with the highest morphological realism.

[0048] 3.3 Compare the original size of the target 3D model with the physical size of the bounding box of the point cloud instance object, calculate the non-uniform scaling matrix in the 3D coordinate system, and apply the matrix to the target 3D model to complete the size normalization.

[0049] Step 4: Import the resized target 3D model into the virtual reality rendering engine. Combine the edge constraint information of the scene spatial relationship graph to calculate the instantiation transformation matrix of each model in the engine coordinate system, thereby completing the automated construction of the 3D scene physical layout and the binding of hierarchical node logical relationships.

[0050] 4.1 The normalized target 3D model is loaded into the virtual reality engine memory through a cross-platform interface, and a unique identifier is assigned to each model instance to facilitate subsequent rendering calls and hierarchical attribute control.

[0051] 4.2 Based on the relative deflection angle and translation distance defined in the scene space relationship diagram, construct quaternions and displacement vectors to solve the absolute transformation matrix of each model instance in the engine world coordinate system and update its space transformation components.

[0052] 4.3 Based on the inclusion and dependency relationships between target objects, establish a parent-child node hierarchy in the scene tree of the virtual reality engine to ensure that the linkage displacement and rotation operations of complex composite objects conform to the physical constraints of the real world.

[0053] Step 5: Perform physical collision detection and clipping repair on the initially constructed 3D virtual scene. At the same time, configure the lighting baking parameters and material reflection properties of the scene environment based on global semantic information. Finally, output a standardized virtual reality scene resource file that meets the needs of immersive interaction.

[0054] 5.1 Attach physical collider components to the entity models in the scene tree and use a ray detection algorithm to scan the overlapping areas of the models. For entities that have mesh penetration, perform a spatial backoff translation algorithm to eliminate unreasonable physical collision interference.

[0055] 5.2 Automatically load skyboxes and global illumination presets based on scene context. Utilize a physically based rendering pipeline to give the model surface the correct diffuse and specular material properties, thereby enhancing the visual expressiveness of virtual reality scenes.

[0056] 5.3 Summarize all mesh assets and material lighting data within the scene, perform serialization and packaging of rendering resources, and finally generate an independent executable program or digital asset package that can be directly read and run by virtual reality terminal devices.

[0057] The invention will be further described below with reference to specific examples and accompanying drawings.

[0058] Example

[0059] See Figure 2 In this embodiment, a high-precision 3D mesh model of a 5-axis CNC machine tool containing 8.5 million triangular facets is acquired, and the vertex coordinates and facet index information of the 5-axis CNC machine tool model are analyzed and transformed into an undirected attribute graph structure. Nodes of the graph structure are defined as vertices of the 3D mesh of the 5-axis CNC machine tool model, and edges are defined as topological connections between vertices. A six-layer graph convolutional neural network is constructed, and the spatial 3D coordinates and normal vectors of the nodes are used as initial features to perform six graph convolution operations to extract a hybrid high-dimensional feature matrix containing the local curvature of the 5-axis CNC machine tool spindle and the global topological association of the machine body. The hybrid high-dimensional feature matrix is ​​input into a dual-branch architecture decoupled neural network for feature separation. The first branch network outputs a low-frequency skeleton feature set representing the physical architecture of the 5-axis CNC machine tool column and guide rail through a low-pass filtering aggregation operation. The second branch network outputs a high-frequency detail feature set representing the microscopic undulations of the 5-axis CNC machine tool shell chamfer and lead screw thread through a high-pass filtering operation. A five-layer perceptron model is constructed, and the high-frequency detail feature set is used as the training target to fit a continuous implicit neural representation function. After convergence, the weight parameters are extracted to generate a 256-dimensional implicit encoding compression vector of the high-frequency features of the five-axis CNC machine tool.

[0060] Based on the low-frequency skeleton feature set, the geometric weights of the vertices of the five-axis CNC machine tool mesh are quantized, and the quadratic error metric cost matrix of all edges in the model is calculated. Mesh edges are sorted in ascending order of error cost to construct an edge folding priority queue. A vertex retention rate threshold of 5% is set for the five-axis CNC machine tool mesh, and the edges with the lowest error cost are extracted from the priority queue. Edge folding and topology shrinking operations are performed to update the 3D coordinates and adjacency structure of the new vertices until the threshold is met, generating a lightweight basic mesh model of the five-axis CNC machine tool with four levels of resolution. The lightweight basic mesh model of the five-axis CNC machine tool and the implicitly encoded compressed vectors are loaded into the memory of the 3D virtual reality engine. During the real-time rendering pipeline stage, the virtual camera world coordinates are obtained to calculate the spatial straight-line distance and observation deflection angle from the viewpoint to the five-axis CNC machine tool tool magazine mesh cluster. The viewpoint distance, observation angle, and implicitly compressed vectors are input into the engine inference module for real-time decoding to generate surface elevation displacement data and normal perturbation maps corresponding to the viewpoint level. The computational shader is used to overlay and map the displacement and normal perturbation data vertex by vertex onto the lightweight base mesh of a five-axis CNC machine tool, and then perform dynamic geometric reconstruction and rasterization rendering output.

[0061] The above description is only a preferred embodiment of the present invention. Modifications may be made within the scope defined by the claims of the present invention, but all such modifications shall fall within the protection scope of the present invention.

Claims

1. A method for automatic scene construction in a 3DVR engine based on scene semantic recognition, characterized in that, The method includes the following steps: Step 1: Acquire multimodal environment perception data of the target scene and perform preprocessing operations to construct an initial 3D point cloud map. Then, use a deep neural network to perform point-level semantic segmentation on the point cloud map to obtain scene semantic information with high-precision category labels and confidence features. Step 2: Calculate the geometric boundaries of the target objects based on the extracted scene semantic information and analyze the relative positions and orientations of different objects to construct a scene spatial relationship graph with topological connection attributes, thereby representing the physical dependencies and associations of various entity components in the 3D scene; Step 3: Analyze the semantic features of the nodes in the scene spatial relationship graph and connect them to the pre-built 3D model asset library. Use the matching algorithm to retrieve candidate models that match the semantic tags and dimensions. At the same time, adaptively adjust the grid vertex coordinates and scaling ratio of the candidate models according to the bounding box parameters. Step 4: Import the resized target 3D model into the virtual reality rendering engine and combine it with the edge constraint information of the scene spatial relationship graph to calculate the instantiation transformation matrix of each model in the engine coordinate system, thereby completing the automated construction of the 3D scene physical layout and the binding of hierarchical node logical relationships. Step 5: Perform physical collision detection and clipping repair on the initially constructed 3D virtual scene. At the same time, configure the lighting baking parameters and material reflection properties of the scene environment based on global semantic information. Finally, output a standardized virtual reality scene resource file that meets the needs of immersive interaction.

2. The method for automatic scene construction in a 3DVR engine based on scene semantic recognition according to claim 1, characterized in that, The specific operations in step 1 include: 2.1 Collect environmental multimodal perception data containing depth information and color images, and use bilateral filtering algorithm to eliminate noise interference generated during data acquisition to improve the overall signal-to-noise ratio and quality of the original environmental data; 2.2 Using the camera intrinsic and extrinsic parameter matrix, the pixel coordinates of the two-dimensional color image are mapped to three-dimensional space to establish the mapping relationship between color information and spatial geometric structure, and complete the initial three-dimensional point cloud map stitching and registration work from multiple perspectives; 2.3 Input the stitched 3D point cloud map into the pre-trained 3D semantic segmentation model to extract local geometric features and global topological structure, and output the semantic category prediction results and corresponding probability distribution of each 3D point.

3. The method for automatic scene construction in a 3DVR engine based on scene semantic recognition according to claim 1, characterized in that, Step 2 includes the following specific operations: 3.1 Based on the same category labels in the semantic segmentation results, perform region growing and clustering segmentation on adjacent point cloud clusters to obtain independent instance objects, and use principal component analysis to solve for the minimum bounding box of each instance object; 3.2 Extract the center three-dimensional coordinates and three-axis direction vectors of the bounding box of each instance object to calculate the Euclidean distance and relative deflection angle between each pair of objects, thereby quantifying the position and orientation constraints between spatial entities; 3.3 Generate a scene spatial relationship graph by treating instance objects as nodes and relative position and orientation constraints as directed edges in the graph structure, and solidify the semantic category and bounding box size attributes of the nodes into the spatial relationship graph.

4. The method for automatic scene construction in a 3DVR engine based on scene semantic recognition according to claim 1, characterized in that, Step 3 includes the following specific operations: 4.1 Extract semantic labels from nodes in the spatial relationship graph and convert them into standard retrieval descriptors. Then, perform hash feature indexing in the structured 3D model database to obtain an initial set of candidate 3D models with the same semantic attributes. 4.2 Calculate the cosine similarity between the morphological descriptor of the candidate 3D model and the geometric features of the original point cloud instance object, sort them in descending order of similarity score, and select the best matching target 3D model with the highest morphological realism. 4.3 Compare the original size of the target 3D model with the physical size of the bounding box of the point cloud instance object, calculate the non-uniform scaling matrix in the 3D coordinate system, and apply the matrix to the target 3D model to complete the size normalization.

5. The method for automatic scene construction in a 3DVR engine based on scene semantic recognition according to claim 1, characterized in that, Step 4 includes the following specific operations: 5.1 The normalized target 3D model is loaded into the virtual reality engine memory through a cross-platform interface, and a unique identifier is assigned to each model instance to facilitate subsequent rendering calls and hierarchical attribute control; 5.2 Based on the relative deflection angle and translation distance defined in the scene spatial relationship diagram, construct quaternions and displacement vectors to solve the absolute transformation matrix of each model instance in the engine world coordinate system and update its spatial transformation components; 5.3 Based on the inclusion and dependency relationships between target objects, establish a parent-child node hierarchy in the scene tree of the virtual reality engine to ensure that the linkage displacement and rotation operations of complex composite objects conform to the physical constraints of the real world.

6. The method for automatic scene construction in a 3DVR engine based on scene semantic recognition according to claim 1, characterized in that, Step 5 includes the following specific operations: 6.1 Attach physical collider components to the entity models in the scene tree and use the ray detection algorithm to scan the overlapping areas of the models. Perform a spatial backoff translation algorithm on entities that have mesh penetration to eliminate unreasonable physical collision interference. 6.2 Automatically load skyboxes and global illumination presets based on scene context. Utilize a physically based rendering pipeline to give the model surface the correct diffuse and specular material properties, thereby enhancing the visual expressiveness of virtual reality scenes. 6.3 Summarize all mesh assets and material lighting data within the scene, perform serialization and packaging of rendering resources, and finally generate an independent executable program or digital asset package that can be directly read and run by virtual reality terminal devices.

Images