UE scene localization migration and reconstruction method and system based on Gaussian sputtering

Abstract

The invention discloses a UE scene localization migration and reconstruction method and system based on Gaussian sputtering, and belongs to the technical field of digital scene construction and virtual reality. Track planning is carried out through a virtual camera matrix, a high-definition image sequence is output, camera parameters are extracted synchronously, left and right hand coordinate system conversion is realized by adopting a coordinate system mapping algorithm, and the real-time migration and reconstruction of the UE scene are realized. An initial point cloud is generated in combination with denoising sharpening processing and the depth map, and three-dimensional reconstruction convergence is accelerated; constructing an initial model by using a spherical harmonic function and a covariance matrix, reducing errors by optimizing renderer iteration, exporting a point cloud format, and constructing a real-time rendering framework by means of block sorting and an Alpha mixing technology; and parameters are dynamically adjusted for performance fluctuation, point cloud density information is fused, light and shadow consistency is ensured, and finally a stable high-fidelity scene model is formed. According to the method, the modeling precision and the rendering efficiency of the virtual reality scene are remarkably improved, and reliable support is provided for interactive application.

Description

Technical Field

[0001] This invention relates to the fields of digital scene construction and virtual reality technology, and in particular discloses a method and system for the localization migration and reconstruction of UE scenes based on Gaussian sputtering. Background Technology

[0002] In the fields of digital scene construction and virtual reality, research on how to achieve efficient and high-quality scene migration is of great significance. However, current mainstream scene migration methods often face challenges such as high technical barriers and high operating costs when dealing with complex environments.

[0003] Many solutions rely on manual intervention, requiring a significant amount of time to readjust scene details, or are limited by the format restrictions of specific platforms, resulting in inefficient migration processes.

[0004] A deeper problem lies in the difficulty of these methods to maintain the visual effects of the scene while simultaneously ensuring the automation of the migration and the smoothness of the final presentation, often resulting in distortion or insufficient performance. Focusing on the technical challenges, a core challenge in scene migration is how to accurately restore the visual details and spatial relationships of the original scene, especially the conversion between different coordinate systems. Since different platforms may use different spatial definition methods, such as differences in direction and units, failure to accurately handle this conversion will lead to the displacement of objects in the scene or disproportionate proportions, thus affecting the realism of the overall effect. This complexity of spatial conversion directly affects whether the migrated scene can seamlessly connect to the new environment, becoming a major obstacle in technical implementation. Taking a specific business example, when migrating a virtual game scene from one development environment to another, assuming the original environment defines space using one direction and unit, while the target environment uses a completely different method, if these spatial parameters cannot be accurately converted, characters in the scene may stand in the wrong position, the proportions of buildings may be distorted, and even the lighting effects may be deviated, seriously affecting the user's immersion.

[0005] Therefore, how to achieve high-precision conversion of spatial parameters during migration and ensure lossless restoration of scene details has become a key problem that needs to be solved in this study. Summary of the Invention

[0006] This invention provides a method and system for domestic migration and reconstruction of UE scenes based on Gaussian sputtering, which aims to achieve high-precision conversion of spatial parameters during the migration process and ensure lossless restoration of scene details.

[0007] One aspect of this invention relates to a method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering, comprising the following steps: S100: Plans an automatic roaming trajectory in a virtual reality environment using a virtual camera matrix, outputs original image sequences and video streams from multiple perspectives, extracts camera intrinsic and extrinsic parameters synchronously, and uses a coordinate system mapping algorithm to transform the left-handed coordinate system to the right-handed coordinate system to obtain the aligned image sequence and camera parameters. S200: For the aligned image sequence and camera parameters, perform denoising and sharpening processing, and simultaneously import the depth map normal map to generate the initial point cloud, accelerate the convergence speed, and obtain the preprocessed image sequence and the initial point cloud. S300: The preprocessed image sequence and initial point cloud are input using a 3D reconstruction algorithm. Color and light are expressed by spherical harmonic functions and the covariance matrix controls the shape of the ellipsoid to obtain the initial 3D model. S400: The initial 3D model is iteratively optimized by optimizing the renderer to minimize the error between the rendered image and the original virtual reality environment image. If the optimization error is lower than a preset threshold, the iteration stops and an optimized 3D model is obtained. S500 optimizes the export of 3D models into point cloud standard format, builds a dedicated rendering pipeline in the spatial rendering engine, and uses block sorting technology to process the rendering order to obtain a preliminary rendering framework. S600: Based on the initial rendering framework, Alpha blending technology is incorporated to achieve real-time rendering frame rate, obtain auxiliary collision mesh data, and obtain an interactive and compatible rendering framework. S700: If rendering performance fluctuations are detected in the interactive compatible rendering framework, the block sorting parameters are adjusted, and the point cloud density information is fused to process the consistency of light and shadow, so as to obtain a stable high-fidelity scene model.

[0008] Further, step S100 includes: S110. Plan an automatic roaming trajectory in a virtual reality environment using a virtual camera matrix, and trigger multi-view rendering instructions based on the trajectory data stream to obtain the original image sequence; S120. Extract the projection matrix of the original image sequence, determine the intrinsic and extrinsic parameter matrices of the original image sequence, and obtain the parameter set; S130. Use a transformation operator to perform a spatial transformation on the extrinsic matrix in the parameter set to obtain the aligned camera extrinsic matrix. S140. Perform geometric correction on the original image sequence based on the aligned camera extrinsic matrix to obtain the aligned image sequence and camera parameters.

[0009] Further, step S200 includes: S210. The aligned image sequence is processed by a median filter operator to remove pixel noise, resulting in a denoised image sequence. S220. Extract grayscale gradient features from the denoised image sequence, and use the Laplacian operator to calculate edge contrast to obtain a sharpened image sequence. S230. Simultaneously import the sharpened image sequence into the depth texture and normal vector, and perform back projection through the camera matrix to determine the set of spatial coordinates. S240. Construct geometric topological relationships based on the set of spatial coordinates. If the projection residual is lower than the preset iteration threshold, generate an initial point cloud. S250. The initial point cloud and the sharpened image sequence are sampled and matched, and dense reconstruction is performed through mapping accuracy verification to obtain the preprocessed image sequence and the initial point cloud.

[0010] Further, step S300 includes: S310. Determine the covariance matrix based on the geometric topological relationship between the preprocessed image sequence and the initial point cloud, and obtain the ellipsoidal morphological features corresponding to the covariance matrix. S320. The initial point cloud is spatially constrained by ellipsoidal morphological features, and the local illumination intensity is fitted by spherical harmonic function to determine the spherical harmonic components, thereby obtaining a color and shadow distribution model. S330. Establish a color mapping function based on the color and light distribution model, map the spherical harmonic components to the surface corresponding to the ellipsoidal morphological features, and obtain the initial three-dimensional model surface texture with light and shadow features. S340. Fine-tune the ellipsoid shape by performing surface texture adjustments on the initial 3D model, match the eigenvalues ​​of the covariance matrix with the preset geometric constraints, and obtain the initial 3D model.

[0011] Further, step S400 includes: S410. Obtain the ellipsoidal morphological features and spherical harmonic components contained in the initial 3D model, and perform rasterization processing through the preset viewpoint transformation matrix to obtain the rendered image. S420. Based on the pixel-by-pixel comparison between the rendered image and the original virtual reality environment image, calculate the pixel residual and determine the mean square error of the current iteration cycle. S430. Based on the mean square error, the gradient operator for the ellipsoidal morphological features and spherical harmonic components is calculated using the backpropagation algorithm to obtain the iteration step size for numerical updates. S440. The spatial position, covariance matrix and color and light distribution model of the initial 3D model are numerically corrected by using the iteration step size to obtain the corrected 3D model. S450. If the optimization error generated by correcting the 3D model is lower than the preset threshold, then stop the iteration process and obtain the optimized 3D model.

[0012] Further, step S500 includes: S510. Obtain the center coordinates and covariance matrix of the optimized 3D model, and transform the ellipsoidal morphological features of the optimized 3D model through the attribute mapping mechanism to obtain the standard point cloud format. S520: Perform spatial index construction according to the point cloud standard format, and load the point cloud standard format into the spatial rendering engine using the memory alignment mechanism; S530. In the spatial rendering engine, tile division is performed based on the viewpoint transformation matrix to determine the screen space area corresponding to the tile division. S540: A depth sorting algorithm is used to arrange the rendering order of the point cloud standard format for the screen space region to obtain an ordered vertex stream. S550 performs vertex shading and pixel blending on an ordered vertex stream through a dedicated rendering pipeline to obtain a preliminary rendering framework.

[0013] Further, step S600 includes: S610: Extract pixel color values ​​and vertex attributes from the initial rendering framework, perform linear interpolation on the pixel color values ​​using a preset blending factor, and determine the blended pixel sequence in conjunction with depth cache data; S620: Perform frame rate threshold monitoring on the mixed pixel sequence. If the current rendering frequency is lower than the preset frequency, the mixed pixel sequence is downsampled using a dynamic thinning algorithm to obtain a high-frequency rendering stream. S630. Extract geometric boundary features based on vertex coordinates in the high-frequency rendering stream, and generate a collision voxel set using a topology reconstruction algorithm to obtain auxiliary collision mesh data. S640: Map the auxiliary collision mesh data to the interaction command response matrix, and match the interaction command response matrix with the rendering pipeline state through the compatibility interface to obtain an interactive compatible rendering framework.

[0014] Further, step S700 includes: S710. If rendering performance fluctuations are detected in the interactive compatible rendering framework, adjust the block sorting values ​​according to the geometric boundary features in the depth cache data. S720. Spatial clustering of vertex coordinates is performed using block sorting values ​​to obtain point cloud density information; S730. Align the light source vector with the surface normal during the topology reconstruction process based on the point cloud density information to handle the lighting consistency of the topology. S740 maps the topology structure with consistent lighting and shadows to the original rendering pipeline, and synthesizes a stable high-fidelity scene model in the original rendering pipeline through multi-level progressive texture mapping.

[0015] Another aspect of the present invention relates to a UE scene localization migration and reconstruction system based on Gaussian sputtering, used to implement the above-mentioned UE scene localization migration and reconstruction method based on Gaussian sputtering, comprising: The image sequence and camera parameter acquisition module is used to plan an automatic roaming trajectory in a virtual reality environment through a virtual camera matrix, output the original image sequence and video stream from multiple perspectives, extract the camera intrinsic and extrinsic parameters synchronously, and use a coordinate system mapping algorithm to transform the left-handed coordinate system to the right-handed coordinate system to obtain the aligned image sequence and camera parameters. The preprocessed image sequence and initial point cloud acquisition module is used to perform denoising and sharpening processing on the aligned image sequence and camera parameters, and simultaneously import the depth map normal map to generate the initial point cloud, thereby accelerating the convergence speed and obtaining the preprocessed image sequence and initial point cloud. The initial 3D model acquisition module is used to input preprocessed image sequences and initial point clouds using a 3D reconstruction algorithm, express color and light and shadow using spherical harmonic functions, and control the ellipsoid shape using a covariance matrix to obtain the initial 3D model. The optimized 3D model acquisition module is used to iteratively optimize the initial 3D model through the optimization renderer, minimizing the error between the rendered image and the original virtual reality environment image. If the optimization error is lower than a preset threshold, the iteration stops, and the optimized 3D model is obtained. The initial rendering framework acquisition module is used to export the optimized 3D model into a point cloud standard format, build a dedicated rendering pipeline in the spatial rendering engine, and use block sorting technology to process the rendering order to obtain the initial rendering framework. The interactive compatible rendering framework acquisition module is used to integrate Alpha blending technology into the initial rendering framework, achieve real-time rendering frame rate, acquire auxiliary collision mesh data, and obtain the interactive compatible rendering framework. The stable high-fidelity scene model acquisition module is used to adjust the block sorting parameters and integrate point cloud density information to process lighting and shadow consistency if rendering performance fluctuations are detected in the interactive compatible rendering framework, thereby obtaining a stable high-fidelity scene model.

[0016] The beneficial effects achieved by this invention are as follows: This invention provides a domestic migration and reconstruction method and system for UE scenes based on Gaussian sputtering. Addressing the business scenario problems of automatic roaming trajectory planning and high-fidelity scene modeling in virtual reality environments, it integrates a complete process from multi-view image sequence generation to 3D model optimization, solving key challenges such as image alignment, rendering performance fluctuations, and lighting consistency. This invention plans trajectories using a virtual camera matrix, outputs high-definition image sequences, and simultaneously extracts camera parameters. It employs a coordinate system mapping algorithm to achieve left-handed / right-handed coordinate system transformation, combined with denoising and sharpening processing and depth map generation to generate initial point clouds, accelerating 3D reconstruction convergence. An initial model is constructed using spherical harmonics and covariance matrices, and errors are reduced through renderer iteration optimization. After exporting the point cloud format, a real-time rendering framework is built using block sorting and alpha blending techniques. To address performance fluctuations, parameters are dynamically adjusted and point cloud density information is fused to ensure lighting consistency, ultimately forming a stable high-fidelity scene model. This invention significantly improves the modeling accuracy and rendering efficiency of virtual reality scenes, providing reliable support for interactive applications. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an embodiment of the UE scene localization migration and reconstruction method based on Gaussian sputtering of the present invention; Figure 2 This is a functional block diagram of an embodiment of the UE scene localization migration and reconstruction system based on Gaussian sputtering of the present invention.

[0018] Explanation of icon numbers: 10. Image sequence and camera parameter acquisition module; 20. Preprocessed image sequence and initial point cloud acquisition module; 30. Initial 3D model acquisition module; 40. Optimized 3D model acquisition module; 50. Preliminary rendering framework acquisition module; 60. Interactive compatible rendering framework acquisition module; 70. Stable high-fidelity scene model acquisition module. Detailed Implementation

[0019] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0020] like Figure 1 As shown, the first embodiment of this invention proposes a domestic migration and reconstruction method for UE scenes based on Gaussian sputtering. The core of this method is to achieve domestic migration and high-fidelity reconstruction of UE (Unreal Engine) scenes through virtual reality environment image acquisition, preprocessing, 3D reconstruction, model optimization, rendering framework construction, and performance tuning. This solves problems such as model distortion, rendering stuttering, interaction incompatibility, and poor lighting effects in traditional UE scene migration processes. It is compatible with domestic hardware and software environments and can be widely applied in fields such as digital twins, virtual reality, and game development. The method includes the following steps: Step S100: Plan the automatic roaming trajectory in the virtual reality environment using a virtual camera matrix, output the original image sequence and video stream from multiple perspectives, extract the camera intrinsic and extrinsic parameters synchronously, and use a coordinate system mapping algorithm to transform the left-handed coordinate system to the right-handed coordinate system to obtain the aligned image sequence and camera parameters.

[0021] In a domestically developed virtual reality environment, a virtual camera matrix is ​​deployed, and an automatic roaming trajectory covering the entire UE scene is planned. The virtual cameras are controlled to move along the preset trajectory, and the original image sequences and video streams are output synchronously from multiple perspectives. During the acquisition process, the intrinsic and extrinsic parameters of each virtual camera are extracted synchronously. A domestically adapted coordinate system mapping algorithm is used to convert the default left-handed coordinate system of the UE scene into the right-handed coordinate system commonly used in domestic software, eliminating model offset caused by coordinate system differences. Finally, the image sequence and standardized camera parameters after coordinate system alignment are obtained, providing a unified data foundation for subsequent image preprocessing and 3D reconstruction.

[0022] This step defines new terms separately: Gaussian sputtering: The core technology of this method refers to mapping 3D points in a scene into Gaussian points with specific ellipsoidal shapes. Color and light information are expressed through spherical harmonic functions, realizing high-fidelity reconstruction and real-time rendering of the scene. Unlike traditional point cloud reconstruction, it has the advantages of fast convergence speed and high light and shadow reproduction, and is suitable for the needs of domestic 3D reconstruction.

[0023] UE scene: refers to a virtual reality scene built on Unreal Engine, which includes core elements such as scene model, lighting effects, and interaction logic. It is the target object of this method for localization migration and reconstruction.

[0024] Domestic migration: refers to the process of adapting a UE scene built on the foreign Unreal Engine to domestic hardware (such as domestic CPUs and graphics cards) and domestic software (such as domestic rendering engines and operating systems) to ensure that the scene can run stably and render efficiently in a domestic environment.

[0025] Virtual camera matrix: refers to a data acquisition system formed by deploying multiple virtual cameras in a preset array. It can simultaneously acquire image and video data of the UE scene from different perspectives, ensuring full coverage of the acquired data. It is the core carrier of multi-view image acquisition.

[0026] Automatic roaming trajectory: refers to the pre-planned path used to control the movement of the virtual camera, covering all key areas of the UE scene, ensuring that multi-view acquisition is not missed, and the trajectory can be dynamically adjusted according to the complexity of the scene.

[0027] Camera intrinsic parameters: These refer to the inherent parameters of the virtual camera itself, including focal length, pixel size, principal point coordinates, etc. They are used to describe the imaging characteristics of the camera and are the core parameters for the transformation between image coordinates and world coordinates.

[0028] Camera extrinsic parameters: These refer to the position and orientation parameters of the virtual camera in the world coordinate system, including translation vectors and rotation matrices, which are used to determine the spatial position of the camera in the scene and achieve spatial alignment of multi-view images.

[0029] Coordinate system mapping algorithm: refers to the mathematical algorithm used to realize the transformation between different coordinate systems. In this embodiment, it is used to convert the default left-handed coordinate system of the UE scene to the right-handed coordinate system commonly used by domestic software, so as to eliminate model offset and rendering errors caused by coordinate system differences and adapt to the domestic development environment.

[0030] Left-handed coordinate system: The default coordinate system used in UE scenes, with the X-axis pointing to the right, the Y-axis pointing upwards, and the Z-axis pointing forwards, which conforms to the conventional design specifications of foreign game engines.

[0031] Right-handed coordinate system: A commonly used coordinate system in domestic software (such as domestic rendering engines and 3D modeling software). Its X-axis is to the right, Y-axis is forward, and Z-axis is upward. It has axial differences from the left-handed coordinate system and needs to be aligned through a mapping algorithm.

[0032] The threshold, specific numerical range, and basis for determining the value in this step are as follows: 1. Virtual camera matrix parameters: Number of cameras: 8-32, preferably 16-24; Camera spacing: 0.5~5m, adjusted according to the UE scene size. When the scene size is ≤100m×100m, take 0.5~2m, and when the scene size is >100m×100m, take 2~5m. Value selection criteria: 8~32 cameras can achieve full coverage of the UE scene and avoid blind spots in the acquisition; 16~24 cameras balance acquisition efficiency and data volume and adapt to the data processing capabilities of domestic hardware; the camera spacing is adjusted according to the scene size to ensure that the images acquired by adjacent cameras have a reasonable overlapping area (overlap rate ≥30%), providing sufficient matching features for subsequent 3D reconstruction and meeting the engineering requirements of multi-view 3D reconstruction.

[0033] 2. Automatic roaming trajectory parameters: Trajectory length: 100~1000m, matching the size of the UE scene; Roaming speed: 1~5m / s, preferably 2~3m / s; Sampling interval: 0.1~0.5m, that is, one frame of image is collected every 0.1~0.5m of movement; The values ​​are determined based on the following criteria: the trajectory length matches the scene size to ensure coverage of all key areas of the scene; the roaming speed of 1~5m / s balances acquisition efficiency and image clarity, avoiding image blurring due to excessive speed; and the sampling interval of 0.1~0.5m ensures sufficient overlap between adjacent frames, improving the accuracy of subsequent point cloud generation and conforming to the technical specifications for multi-view image acquisition.

[0034] 3. Image and video stream parameters: Image resolution: 1920×1080~4096×2160 pixels, preferably 2560×1440 pixels; Video stream frame rate: 24~34fps, preferably 30fps; Original image sequence length: 1000~10000 frames, adjusted according to scene complexity, preferably 4000 frames; Value selection criteria: A resolution of 1920×1080~4096×2160 pixels can balance image clarity with the processing capabilities of domestically produced hardware, while 2560×1440 pixels is the optimal value to avoid excessive data volume and processing lag due to excessively high resolution; a frame rate of 24~34fps can ensure smooth video streaming, while 30fps is suitable for the real-time processing requirements of domestically produced rendering engines; a sequence of 1000~10000 frames of images can meet the reconstruction needs of scenes with different complexities and ensure the detail integrity of the reconstructed model.

[0035] 4. Camera parameter value range: Camera internal parameters: focal length 35~100mm, pixel size 1~5μm, principal point coordinate deviation ≤1 pixel; Camera extrinsic parameters: translation vector range (-1000~1000m, -1000~1000m, -1000~1000m), rotation matrix angle range (-180°~180°), parameter error ≤0.1%; Value selection criteria: a focal length of 35~100mm can cover the acquisition needs of different perspectives, a pixel size of 1~5μm ensures image imaging accuracy, and a principal point coordinate deviation of ≤1 pixel can reduce imaging distortion; the range of camera extrinsic parameters matches the UE scene size, and an error of ≤0.1% can ensure the spatial alignment accuracy of multi-view images, avoid model offset due to parameter errors, and meet the parameter requirements of 3D reconstruction.

[0036] 5. Coordinate system mapping threshold: The mapping error is ≤0.01m, meaning the position deviation of the corresponding point in the mapped coordinate system is ≤0.01m. The mapping time is ≤100ms, ensuring that the coordinate system transformation is completed quickly and does not affect the efficiency of subsequent data processing; The values ​​are determined based on the following criteria: a mapping error of ≤0.01m ensures the accuracy of coordinate system alignment and avoids distortion of the subsequent 3D model due to alignment deviation; a mapping time of ≤100ms adapts to the processing speed of domestic hardware and meets the needs of real-time data processing, and is determined in combination with the complexity of the coordinate system mapping algorithm.

[0037] Step S200: For the aligned image sequence and camera parameters, perform denoising and sharpening processing, and simultaneously import the depth map normal map to generate the initial point cloud, accelerate the convergence speed, and obtain the preprocessed image sequence and the initial point cloud.

[0038] For the coordinate-aligned image sequence and camera parameters obtained in step S100, a domestically adapted image denoising and sharpening algorithm is used to remove noise generated during image acquisition (such as Gaussian noise and salt-and-pepper noise), enhance the clarity of image details, and obtain a preprocessed image sequence. Simultaneously, the original depth map and normal map of the UE scene are imported, and combined with the preprocessed image sequence and camera parameters, an initial point cloud is generated using the Gaussian sputtering initialization method to accelerate the convergence speed of subsequent 3D reconstruction algorithms. Finally, the preprocessed image sequence and initial point cloud are obtained, providing high-quality data input for 3D reconstruction.

[0039] This step defines new terms separately: Denoising and sharpening: This refers to the process of removing noise and enhancing details in the acquired image sequence. Denoising algorithms eliminate image noise, while sharpening algorithms enhance image edges and texture details, thereby improving image quality and providing clear feature data for subsequent 3D reconstruction.

[0040] Depth map: refers to the image that records the distance information from each pixel in the UE scene to the virtual camera. The pixel value corresponds to the distance and is the core auxiliary data for generating the initial point cloud and restoring the three-dimensional structure of the scene.

[0041] Normal mapping: refers to an image that records the direction of normals on the surface of an object in a UE scene. It is used to represent the concave and convex shape of the object's surface, which can enhance the detail of the 3D model and improve the realism of the reconstructed scene.

[0042] Initial point cloud: refers to the set of 3D points generated by Gaussian sputtering based on preprocessed image sequences, camera parameters, depth maps and normal maps. Each point contains information such as spatial coordinates, color and normals. It is the initial data for 3D reconstruction and can speed up the convergence of reconstruction algorithms.

[0043] Convergence speed: refers to the speed at which a 3D reconstruction algorithm generates a stable 3D model by iteratively optimizing from an initial point cloud. The quality of the initial point cloud directly affects the convergence speed; a high-quality initial point cloud can significantly shorten the iteration time.

[0044] The threshold, specific numerical range, and basis for determining the value in this step are as follows: 1. Noise reduction and sharpening parameters: Noise removal thresholds: Gaussian noise variance ≤ 0.01, salt-and-pepper noise density ≤ 1%, and noise residue rate after treatment ≤ 0.5%; Sharpening intensity: 0.5~2.0, preferably 1.0~1.5; Processing time: Single frame image processing time ≤ 20ms, full sequence processing time ≤ 300s; Value selection criteria: Gaussian noise variance ≤ 0.01, salt-and-pepper noise density ≤ 1%, which can effectively remove common noise generated during image acquisition, and the noise residue rate after processing is ≤ 0.5% to ensure image quality; sharpening intensity of 0.5~2.0 can balance detail enhancement and image distortion, and 1.0~1.5 is the optimal value to avoid over-sharpening and image artifacts; single frame processing time ≤ 20ms, full sequence ≤ 300s, adapting to the processing capabilities of domestic hardware and meeting the efficiency requirements of actual engineering.

[0045] 2. Depth map and normal map parameters: Depth map resolution: consistent with image sequence resolution (1920×1080~4096×2160 pixels), depth measurement range 0.1~1000m, depth accuracy ±0.01m; Normal map resolution: consistent with the image sequence resolution, normal direction accuracy ±0.1°; Value selection criteria: The resolution of the depth map and normal map is consistent with the image sequence to ensure data matching and avoid distortion of point cloud generation due to resolution differences; the depth measurement range of 0.1~1000m covers the conventional depth requirements of UE scenarios, and the depth accuracy of ±0.01m and normal direction accuracy of ±0.1° can ensure the detail accuracy of the initial point cloud and meet the auxiliary data requirements of 3D reconstruction.

[0046] 3. Initial point cloud parameters: Point cloud density: 100~1000 points / ㎡, preferably 300~500 points / ㎡, adjusted according to the specific needs of the scene; Point cloud error: ≤0.05m, that is, the deviation between the initial point cloud and the actual position of the scene is ≤0.05m; Convergence speed improvement: ≥30%, meaning that compared to reconstruction methods without initial point clouds, the convergence speed is improved by ≥30%; Value selection criteria: A point cloud density of 100~1000 points / ㎡ can balance detail restoration and data volume, while 300~500 points / ㎡ is the optimal value to adapt to the processing capabilities of domestic hardware; a point cloud error of ≤0.05m can ensure the accuracy of the initial point cloud and provide a reliable foundation for subsequent reconstruction; an increase in convergence speed of ≥30% can significantly shorten the overall time of 3D reconstruction, improve engineering efficiency, and meet the core advantages of Gaussian sputtering reconstruction.

[0047] Step S300: Input the preprocessed image sequence and initial point cloud using a 3D reconstruction algorithm, express color and light and shadow using spherical harmonic functions, and control the ellipsoid shape using the covariance matrix to obtain the initial 3D model.

[0048] Using a domestically adapted 3D reconstruction algorithm, the preprocessed image sequence and initial point cloud obtained in step S200 are taken as input. Based on Gaussian sputtering technology, the initial point cloud is mapped to Gaussian points. The color and light information of each Gaussian point is expressed by a spherical harmonic function. The ellipsoidal shape of the Gaussian points is controlled by the covariance matrix to simulate the surface details and light and shadow changes of objects in the scene. Through iterative optimization, the multi-view image features are matched and fused with the Gaussian points to restore the 3D structure of the UE scene, and finally the initial 3D model is obtained, completing the initial reconstruction of the UE scene.

[0049] This step defines new terms separately: 3D reconstruction algorithm: refers to the algorithm used to restore the 3D structure of a scene from multi-view images and point cloud data. This embodiment adopts a 3D reconstruction algorithm based on Gaussian sputtering, which is compatible with domestic hardware and has the advantages of fast convergence speed, high reconstruction accuracy and good light and shadow restoration, which is different from traditional point cloud reconstruction algorithms.

[0050] Spherical harmonic functions are mathematical functions used to express the distribution of light and shadow in three-dimensional space. They can accurately describe the color and brightness changes of objects from different perspectives and are used for modeling the light and shadow information of Gaussian points to improve the light and shadow fidelity of the reconstructed model.

[0051] Covariance matrix: refers to the matrix used to control the shape of the Gaussian point ellipsoid. By adjusting the parameters of the covariance matrix, the length and direction of the major and minor axes of the Gaussian point can be changed, simulating the bumps, curves and other shapes of the object surface in the scene, and improving the model's detail reproduction.

[0052] Gaussian point: refers to the core element of Gaussian sputtering technology. It is a three-dimensional point with a specific ellipsoidal shape that carries color and light and shadow information. A collection of multiple Gaussian points can form a high-fidelity three-dimensional model, which has better light and shadow performance and detail reproduction capabilities compared to traditional point clouds.

[0053] Initial 3D model: refers to the 3D model of the UE scene initially generated by the 3D reconstruction algorithm. It contains the basic 3D structure and lighting features of the scene, but may have problems such as rough details and uneven lighting, which require subsequent iterations and optimizations.

[0054] The threshold, specific numerical range, and basis for determining the value in this step are as follows: 1. 3D reconstruction algorithm parameters: Number of iterations: 1000~5000, preferably 2000~3000; Feature matching threshold: Matching similarity ≥ 85%, that is, the matching similarity between multi-view image features and Gaussian points ≥ 85%; Reconstruction time: ≤120min, preferably ≤90min; The criteria for setting the values ​​are as follows: 1000~5000 iterations can ensure the stability of the reconstructed model, 2000~3000 iterations can balance reconstruction accuracy and efficiency, avoiding model distortion due to too few iterations and excessive time consumption due to too many iterations; a matching similarity of ≥85% can ensure the accuracy of feature matching and reduce model defects caused by incorrect matching; and a reconstruction time of ≤120min can adapt to the processing capabilities of domestic hardware and meet the efficiency requirements of actual engineering.

[0055] 2. Spherical harmonic function parameters: The order of spherical harmonic functions: 2 to 6, preferably 3 to 4; Lighting and shadow restoration accuracy: ≥90%, meaning the similarity between the lighting and shadow effects of the reconstructed model and the original UE scene is ≥90%. The selection criteria are as follows: 2nd to 6th order spherical harmonic functions can balance the accuracy of light and shadow restoration and the amount of computation. 3rd to 4th order is the optimal value, which can accurately express the changes in light and shadow without causing a surge in computation due to excessive order, and is compatible with domestic hardware; light and shadow restoration accuracy ≥90% can ensure that the light and shadow effect of the reconstructed model is consistent with the original UE scene, meet the requirements of high-fidelity reconstruction, and is determined in combination with the expressive power of spherical harmonic functions.

[0056] 3. Covariance matrix and Gaussian point parameters: The range of values ​​for the covariance matrix is: 0.01~0.5m² for diagonal elements and -0.1~0.1m² for off-diagonal elements; The major axis of the Gaussian point ellipsoid ranges from 0.01 to 0.5 m, and the minor axis ranges from 0.005 to 0.25 m. Number of Gaussian points: 10 6 ~10 8 One, adjusted according to the complexity of the scenario; Value selection criteria: The diagonal elements of the covariance matrix control the size of the ellipsoid, while the off-diagonal elements control its tilt. The value range of 0.01~0.5m² can simulate details of objects of different sizes. The major and minor axes of the Gaussian point ellipsoid match the dimensions of the scene details, ensuring accurate detail reproduction. 6 ~10 8 Each Gaussian point can cover UE scenarios of varying complexity, balancing detail reproduction with data processing efficiency, and adapting to the storage and processing capabilities of domestically produced hardware.

[0057] 4. Initial 3D model parameters: Model accuracy: ≤0.1m, meaning the positional deviation between the initial 3D model and the original UE scene is ≤0.1m; Detail reproduction rate: ≥80%, meaning that the reproduction rate of key details (such as object edges and textures) in the original UE scene in the initial model is ≥80%; Value criteria: Model accuracy ≤ 0.1m ensures the overall accuracy of the initial model and avoids significant deviations; detail restoration rate ≥ 80% ensures that the initial model retains the core details of the original scene, providing a good foundation for subsequent optimization and meeting the initial requirements of high-fidelity reconstruction.

[0058] Step S400: Iteratively optimize the initial 3D model using an optimization renderer to minimize the error between the rendered image and the original virtual reality environment image. If the optimization error is lower than a preset threshold, the iteration stops, and an optimized 3D model is obtained.

[0059] A domestically produced optimized renderer is used to iteratively optimize the initial 3D model obtained in step S300. Using the image in the original virtual reality environment as a reference standard, the error between the rendered image and the original image (such as pixel error and lighting error) is calculated. By adjusting the ellipsoid shape of the Gaussian point, color and lighting parameters, model details, etc., the error is gradually minimized. A preset optimization error threshold is set. If the optimization error is lower than the threshold during the iteration process, the iteration is stopped. Finally, an optimized 3D model is obtained, which solves the problems of rough details, uneven lighting, and large deviation from the original scene in the initial model.

[0060] This step defines new terms separately: Optimized renderer: refers to a domestically produced software module used for rendering and iterative optimization of 3D models. It supports Gaussian point rendering, lighting and shadow adjustment, and detail optimization. It can accurately calculate the error between the rendered image and the original image, realize iterative optimization of the model, and adapt to domestic hardware environments.

[0061] Iterative optimization refers to the process of gradually reducing rendering errors by repeatedly adjusting model parameters and rendering images. It is the core link in improving the fidelity of 3D models. Each iteration adjusts parameters based on the error results of the previous iteration.

[0062] Optimization error: refers to the difference between the rendered image output by the optimized renderer and the original virtual reality environment image, including pixel error, lighting error, contour error, etc., and is the core indicator for judging the optimization effect of the model.

[0063] Optimized 3D model: refers to a 3D model whose rendering error is lower than a preset threshold after iterative optimization. Its details, lighting, and outlines are highly consistent with the original UE scene, meeting the requirements of high-fidelity reconstruction and serving as the foundation for subsequent rendering framework construction.

[0064] The threshold, specific numerical range, and basis for determining the value in this step are as follows: 1. Iterative optimization of parameters: Number of iterations: 500~3000 times, with a preferred number of 1000~2000 times; Time taken per iteration: ≤500ms, total optimization time: ≤30min; The selection criteria are as follows: 500-3000 iterations can ensure the optimization effect, 1000-2000 iterations can balance the optimization accuracy and efficiency, avoiding insufficient optimization due to too few iterations and excessive time consumption due to too many iterations; the time consumed by each iteration is ≤500ms and the total time is ≤30min, which is suitable for the processing capabilities of domestic hardware and meets the efficiency requirements of actual engineering.

[0065] 2. Optimize the error threshold and its selection criteria: Pixel error threshold: ≤5%, meaning the pixel difference between the rendered image and the original image is ≤5%; Lighting and shadow error threshold: ≤8%, meaning the difference in lighting and shadow brightness between the rendered image and the original image is ≤8%; Overall optimization error threshold: ≤6%, that is, the weighted average of pixel error and lighting error is ≤6%. The iteration can be stopped when any single error is lower than the corresponding threshold and the overall error is lower than 6%. Values ​​are determined based on the following criteria: a pixel error of ≤5% ensures that the details of the rendered image are consistent with the original image, avoiding obvious pixel deviations; a lighting and shadow error of ≤8% ensures that the lighting and shadow effects of the rendered image match the original scene, avoiding lighting and shadow distortion; and a comprehensive error of ≤6% can balance the optimization effects of details and lighting and shadow, meeting the core requirements of high-fidelity reconstruction. This is determined by combining the accuracy capabilities of domestic renderers, while ensuring that the optimized model can adapt to the real-time rendering needs of domestic hardware.

[0066] 3. Optimize 3D model parameters: Model accuracy: ≤0.05m, meaning the positional deviation between the optimized model and the original UE scene is ≤0.05m; Detail reproduction rate: ≥95%, meaning that the proportion of key details in the original UE scene reproduced in the optimized model is ≥95%; Lighting and shadow similarity: ≥95%, meaning the similarity between the lighting and shadow effects of the optimized model and the original UE scene is ≥95%; The criteria for determining the values ​​are: model accuracy ≤ 0.05m, detail restoration rate ≥ 95%, and lighting and shadow similarity ≥ 95%. This ensures that the optimized model meets high fidelity requirements, is highly consistent with the original UE scene, meets the scene restoration requirements after domestic migration, and is determined in combination with the technical goals of iterative optimization.

[0067] Step S500: Export the optimized 3D model as a point cloud standard format, build a dedicated rendering pipeline in the spatial rendering engine, and use block sorting technology to process the rendering order to obtain a preliminary rendering framework.

[0068] The optimized 3D model obtained in step S400 is exported to a point cloud standard format supported by the domestic rendering engine to ensure that the model can be imported and rendered normally in the domestic environment. In the domestic spatial rendering engine, a dedicated rendering pipeline adapted to Gaussian sputtering technology is built, integrating core modules such as model rendering, lighting and shadow calculation, and detail processing. The block sorting technology is used to divide the Gaussian points in the scene into blocks and sort them according to certain rules to optimize the rendering order, reduce rendering lag, and improve rendering efficiency. Finally, a preliminary rendering framework is obtained, which provides a foundation for subsequent interactive compatibility optimization.

[0069] This step defines new terms separately: Point cloud standard format: refers to the standardized file format used to store point cloud data that is supported by domestic rendering engines. In this embodiment, PLY and PCD formats are preferred (both are compatible with domestic software) to ensure that the optimized 3D model can be successfully imported into the domestic rendering engine and achieve cross-platform compatibility.

[0070] Spatial rendering engine: refers to the core module of domestically produced software used to realize real-time rendering of 3D scenes. It supports functions such as Gaussian point rendering, light and shadow calculation, and interactive processing. It is the core carrier for building rendering framework and realizing scene visualization.

[0071] Dedicated rendering pipeline: refers to a customized rendering workflow built for Gaussian sputtering technology, including core steps such as Gaussian point import, lighting and shadow calculation, pixel rendering, and detail optimization. It is adapted to the rendering characteristics of Gaussian points, improves rendering efficiency and fidelity, and is different from traditional point cloud rendering pipelines.

[0072] Block sorting technology: This refers to dividing the Gaussian points in the scene into blocks according to their spatial location and rendering priority, sorting the Gaussian points in each block, optimizing the rendering order (such as sorting them from near to far from the camera), reducing resource consumption during rendering, avoiding stuttering, and improving rendering efficiency.

[0073] Preliminary rendering framework: This refers to the basic rendering framework consisting of a dedicated rendering pipeline, Gaussian points sorted by block, and basic rendering parameters. It can achieve basic rendering of optimized 3D models, but it does not yet have interactive compatibility capabilities and requires further optimization.

[0074] The threshold, specific numerical range, and basis for determining the value in this step are as follows: 1. Point cloud export parameters: Export formats: PLY and PCD formats (compatible with domestic rendering engines); Export accuracy: ≤0.01m, meaning the deviation between the exported point cloud data and the optimized 3D model is ≤0.01m; Export time: ≤60s, preferably ≤30s; Value selection criteria: PLY and PCD formats are point cloud standard formats widely supported by domestic rendering engines, ensuring that exported point clouds can be successfully imported; export accuracy ≤0.01m ensures the accuracy of point cloud data and avoids model distortion during the export process; export time ≤60s adapts to the processing efficiency of domestic hardware and meets actual engineering needs.

[0075] 2. Dedicated rendering pipeline parameters: Rendering pipeline frame rate: ≥30fps, preferably ≥60fps; Lighting and shadow calculation latency: ≤20ms; Value criteria: A frame rate of ≥30fps ensures smooth rendering, ≥60fps adapts to the rendering needs of high-fidelity scenes and meets the performance requirements of domestic rendering engines; a light and shadow calculation latency of ≤20ms ensures the real-time performance of light and shadow rendering, avoids light and shadow stuttering, and improves user experience.

[0076] 3. Block sorting technical parameters: Block size: 0.5~5m×0.5~5m×0.5~5m, preferably 1~2m×1~2m×1~2m, adjust according to scene size; Sorting delay: ≤10ms, meaning the time taken to complete block sorting is ≤10ms; Rendering efficiency improvement: ≥40%, meaning that compared to non-block sorting, rendering efficiency is improved by ≥40%; Value selection criteria: A chunk size of 0.5~5m can balance rendering efficiency and detail, with 1~2m being the optimal value to avoid rendering stuttering due to excessively large chunks and a surge in computational load due to excessively small chunks; a sorting latency of ≤10ms can ensure that sorting does not affect real-time rendering; a rendering efficiency improvement of ≥40% can significantly optimize rendering performance, adapt to the processing capabilities of domestic hardware, and be determined in combination with the optimization goals of chunk sorting technology.

[0077] Step S600: Integrate Alpha blending technology into the initial rendering framework to achieve real-time rendering frame rate, obtain auxiliary collision mesh data, and obtain an interactive and compatible rendering framework.

[0078] Based on the preliminary rendering framework obtained in step S500, the domestically adapted Alpha blending technology is integrated to optimize the rendering effect of semi-transparent objects (such as glass and smoke) in the scene, while ensuring that the rendering frame rate is stable within the preset range to achieve real-time rendering. Simultaneously, the geometric information of the 3D model is extracted and optimized to generate auxiliary collision mesh data, which is used to realize interactive functions in the scene (such as click and collision detection), solve the problem of incompatibility between rendering and interaction, and finally obtain an interactively compatible rendering framework to achieve both scene visualization and interactive functions.

[0079] This step defines new terms separately: Alpha blending technology refers to a technique used to render semi-transparent objects in a scene. By adjusting the alpha channel (transparency) of an object, it achieves a natural blending of the semi-transparent object with the background and other objects, enhancing the realism of the rendered image. It is adapted to the rendering characteristics of Gaussian sputtering and is a core optimization technology in domestic rendering.

[0080] Real-time rendering frame rate: refers to the number of frames of rendered images output by the rendering engine per second. It is a core indicator for measuring rendering performance. The higher the frame rate, the smoother the rendered images. The real-time rendering frame rate must meet the needs of scene interaction.

[0081] Auxiliary collision mesh data: refers to simplified mesh data generated based on the geometry of an optimized 3D model for collision detection. Compared with the original model mesh, it has the advantages of small data volume and fast detection speed. It is used to realize interactive collision detection in the scene and ensure the smoothness of interactive functions.

[0082] Interactive compatible rendering framework: This refers to a rendering framework that incorporates Alpha blending technology and auxiliary collision mesh data on the basis of a preliminary rendering framework. It can realize real-time rendering, semi-transparent effect optimization and interactive functions, solve the problem of incompatibility between rendering and interaction, and adapt to the interactive needs of domestic products.

[0083] The threshold, specific numerical range, and basis for determining the value in this step are as follows: 1. Alpha Hybridization Technology Parameters: Alpha channel value range: 0~1, where 0 represents complete transparency and 1 represents complete opacity; Mixing precision: ≤0.01, meaning the Alpha value adjustment precision is ≤0.01; Semi-transparent rendering error: ≤3%, meaning the difference between the rendering effect of semi-transparent objects and the original UE scene is ≤3%; Value selection criteria: Alpha channel 0~1 can achieve different levels of transparency adjustment to meet the rendering needs of different semi-transparent objects; blending precision ≤0.01 can ensure the fineness of transparency adjustment and avoid obvious transparency banding; semi-transparent rendering error ≤3% can ensure that the semi-transparent effect is consistent with the original UE scene and improve rendering fidelity, which is determined in combination with the rendering characteristics of Alpha blending technology.

[0084] 2. Real-time rendering frame rate threshold and its determination basis: Real-time rendering frame rate: ≥30fps, preferably ≥60fps, and no less than 24fps; Frame rate fluctuation: ≤5fps, meaning the frame rate difference between consecutive frames is ≤5fps; Value selection criteria: A frame rate of ≥30fps ensures smooth rendering and meets scene interaction requirements; ≥60fps adapts to the experience requirements of high-fidelity scenes; and a minimum of 24fps can avoid screen stuttering; a frame rate fluctuation of ≤5fps can ensure stable rendering and avoid visual discomfort caused by sudden increases or decreases in frame rate. The values ​​are determined based on the performance of the domestic rendering engine and the scene interaction requirements.

[0085] 3. Auxiliary collision mesh data parameters: Mesh simplification ratio: 50%~80%, meaning the data volume of the auxiliary collision mesh is 50%~80% of the original model mesh; Collision detection accuracy: ≤0.05m, meaning the positional deviation of the collision detection is ≤0.05m; Collision detection latency: ≤10ms; The values ​​are determined based on the following criteria: a grid simplification ratio of 50% to 80% can reduce the amount of data and improve the detection speed while ensuring collision detection accuracy; a collision detection accuracy of ≤0.05m can ensure the accuracy of interaction and avoid collision detection errors; a collision detection latency of ≤10ms can ensure smooth interaction response and meet the real-time requirements of scene interaction, and is determined in combination with the processing capabilities of domestic hardware.

[0086] Step S700: If rendering performance fluctuations are detected in the interactive compatible rendering framework, adjust the block sorting parameters, fuse point cloud density information to process lighting consistency, and obtain a stable high-fidelity scene model.

[0087] The rendering performance of the interactive compatible rendering framework obtained in step S600 is monitored in real time. If rendering performance fluctuations are detected (such as frame rate drop or stuttering), the block sorting parameters (such as block size and sorting rules) are adjusted to optimize the allocation of rendering resources. At the same time, point cloud density information is integrated to adjust the lighting and shadow parameters of different areas in the scene and handle lighting and shadow consistency issues (such as uneven lighting and shadow, and lighting and shadow misalignment) to ensure that the lighting and shadow effects of the scene are uniform. After performance tuning and lighting and shadow optimization, a stable and high-fidelity scene model is obtained, and the localization migration and reconstruction of the UE scene is completed, ensuring that the model can run stably, efficiently and with high fidelity in the localized environment.

[0088] This step defines new terms separately: Rendering performance fluctuations refer to abnormal performance conditions such as frame rate drops, increased rendering latency, and screen stuttering that occur during the operation of the interactive compatible rendering framework. These are mainly caused by factors such as scene complexity, unreasonable resource allocation, and excessive lighting and shadow calculations.

[0089] Point cloud density information: refers to the distribution density data of Gaussian points in different areas of the scene. The higher the density, the richer the scene details, but the greater the rendering computation. Rendering resource allocation and lighting parameters can be adjusted based on point cloud density information.

[0090] Lighting consistency: refers to the uniformity of lighting effects in different areas and on different objects in a scene, without obvious unevenness in brightness, misalignment of light and shadow, or abrupt changes in light and shadow. It is one of the core requirements of high-fidelity scenes.

[0091] Stable high-fidelity scene model: This refers to a UE scene reconstruction model that has stable rendering performance, high detail reproduction, consistent lighting effects, and can run stably in a domestic environment after performance tuning and lighting optimization. It is the core output of this method and realizes the goal of domestic migration and high-fidelity reconstruction of UE scenes.

[0092] The threshold, specific numerical range, and basis for determining the value in this step are as follows: 1. Threshold for judging rendering performance fluctuations and the basis for determining its value: Frame rate fluctuation threshold: >5fps, that is, the frame rate difference between consecutive frames is >5fps, which is judged as performance fluctuation; Minimum frame rate threshold: <24fps, that is, a frame rate below 24fps is considered a performance fluctuation; Rendering latency threshold: >50ms, meaning that a rendering latency exceeding 50ms is considered a performance fluctuation; Values ​​are determined based on the following criteria: frame rate fluctuations > 5fps and minimum frame rates < 24fps will cause screen stuttering and affect user experience; rendering latency > 50ms will cause slow interactive response, which does not meet the requirements of real-time interaction; the above performance fluctuation judgment thresholds are determined by combining the performance of domestic rendering engines and the usage requirements of scenarios to ensure that performance problems are detected and dealt with in a timely manner.

[0093] 2. Adjustment range of block sorting parameters: The block size adjustment range is 0.5~5m×0.5~5m×0.5~5m, and can be adjusted by ±0.5~1m based on the original block size; Sorting rules: Prioritize sorting by "distance from camera to camera", and switch to "block load balancing" sorting when performance fluctuates; Value selection criteria: The block size adjustment range is consistent with the block size of the initial rendering framework. The adjustment range of ±0.5~1m can accurately optimize rendering performance and avoid rendering distortion caused by excessive adjustment; the sorting rule switching can be optimized in a targeted manner according to the reasons for performance fluctuations (such as long-distance rendering lag, uneven block load), adapt to different performance abnormal scenarios, and be determined in combination with the optimization logic of block sorting technology.

[0094] 3. Point cloud density and lighting consistency parameters: Point cloud density adjustment range: ±10%~±30%, meaning that the point cloud density of the corresponding area can be adjusted by ±10%~±30% based on performance fluctuations. Lighting and shadow consistency error: ≤2%, meaning the difference in lighting and shadow brightness between different areas of the scene is ≤2%; Lighting and shadow adjustment delay: ≤15ms; Values ​​are determined based on the following criteria: Point cloud density adjustment of ±10% to ±30% can optimize rendering computation and alleviate performance fluctuations while ensuring detail; Lighting and shadow consistency error ≤2% can ensure uniform lighting and shadow effects in the scene and avoid problems such as uneven brightness; Lighting and shadow adjustment delay ≤15ms can ensure that lighting and shadow optimization does not affect real-time rendering and meets the requirements of high-fidelity scenes. The values ​​are determined by combining the characteristics of point cloud density and lighting and shadow rendering.

[0095] 4. Stable high-fidelity scene model parameters: Rendering performance: Frame rate is stable at 30~60fps, frame rate fluctuation ≤3fps, rendering latency ≤30ms; Model fidelity: Similarity to the original UE scene ≥ 98%, detail reproduction rate ≥ 98%, lighting and shadow similarity ≥ 98%; Stability: Continuous operation for ≥24 hours without lag or crashes, with performance fluctuations ≤3 times / hour; The values ​​are determined based on the following criteria: a stable frame rate of 30-60fps, fluctuations of ≤3fps, and latency of ≤30ms, which ensures smooth scene rendering and timely interactive response; a model fidelity of ≥98%, which ensures that the reconstructed model is highly consistent with the original UE scene and meets the high fidelity requirements; and continuous operation for ≥24 hours with performance fluctuations of ≤3 times / hour, which ensures that the model runs stably in the domestic environment, meets the core goal of domestic migration, and is determined in combination with the stability requirements of the actual engineering project.

[0096] Furthermore, the UE scene localization migration and reconstruction method based on Gaussian sputtering provided in this embodiment includes step S100 as follows: Step S110: Plan an automatic roaming trajectory in the virtual reality environment using a virtual camera matrix, trigger multi-view rendering instructions based on the trajectory data stream, and obtain the original image sequence.

[0097] The automatic roaming trajectory is obtained using the following formula: (1) In formula (1), For the first The automatic roaming trajectory at any given time, measured in meters (m), originates from the trajectory planning results of the virtual camera matrix and is a three-dimensional coordinate sequence. The value range is the spatial coordinate interval of the virtual reality environment; This is a trajectory generation operator with no unit, used to plan the roaming trajectory based on the virtual camera matrix and the field of view. It uses the B-spline interpolation algorithm to achieve smooth trajectory generation. This is a virtual camera matrix, without units, derived from the virtual reality environment configuration, and is a 4×4 matrix containing the position and orientation parameters of all virtual cameras; This refers to the virtual camera's field of view, measured in degrees (°), derived from camera configuration parameters, with a value range of [value range missing]. ; The timestamp, in seconds (s), is derived from the time series of trajectory planning and takes the value of a non-negative real number. The control logic of formula (1) uses the virtual camera matrix as the spatial reference, the view range as the coverage constraint, and the timestamp as the temporal driver. Through a trajectory generation operator based on B-spline interpolation, it integrates three types of core parameters to generate an autonomous roaming trajectory that is temporally continuous, spatially smooth, and has full view coverage. This formula realizes the autonomous roaming planning of the virtual camera in the UE (Unreal Engine) scene, provides the core driver for the acquisition of multi-view original image sequences, and is the source input guarantee for the localization migration and reconstruction of the UE scene based on Gaussian sputtering.

[0098] In a virtual reality environment, an automatic roaming trajectory is planned using a virtual camera matrix. For example, in a virtual city tour system, the virtual camera matrix is ​​defined as a 4x4 transformation matrix to simulate the position and orientation of the camera in three-dimensional space, thus planning a smooth curved trajectory from the starting point to the ending point, such as a Bézier curve path. Multi-view rendering instructions are triggered based on the trajectory data stream to obtain the original image sequence. Specifically, the trajectory data stream includes position, rotation, and timestamp sequences. This data triggers the rendering engine in real time, such as the multi-view rendering module in the Unity engine, to generate image sequences from different angles (such as front view, side view, and top view). For example, in a virtual reality headset, when the trajectory data stream instructs the camera to move to a specific point, the rendering instructions call the GPU to perform rasterization rendering, generating a series of RGB image frames with a resolution of 1920x1080. These frames capture the dynamic changes of the virtual environment, thus obtaining a complete original image sequence for subsequent processing.

[0099] Step S120: Extract the projection matrix of the original image sequence, determine the intrinsic and extrinsic parameter matrices of the original image sequence, and obtain the parameter set.

[0100] The intrinsic and extrinsic parameter matrices of the original image sequence are determined using the following formulas: (2) In formula (2), This is the camera intrinsic parameter matrix of the original image sequence. It is unitless, extracted from the projection matrix of the original image sequence, and is a 3×3 matrix that represents the camera's internal optical parameters. The focal length of the camera along the x and y axes is expressed in pixels (px). It is derived from the analysis of the projection matrix and its value is a positive integer. These are the principal point coordinates of the camera's imaging plane, in pixels (px), derived from the analysis of the projection matrix, and their values ​​are positive integers. The external parameter matrix of the original image sequence is unitless and is extracted from the projection matrix of the original image sequence. It is a 4×4 matrix that represents the position and orientation of the camera in the world coordinate system. This is a rotation matrix, unitless, derived from the extrinsic matrix partitioning, and is a 3×3 orthogonal matrix. Its values ​​satisfy the following range: ( (the identity matrix); It is a translation vector, with units of m, derived from the external parameter matrix splitting, and is a 3×1 vector. Its value range is the spatial coordinate interval of the virtual reality environment. The zero vector is the transpose of the zero vector, has no unit, and is a 1×3 zero vector. The control logic of formula (2) is to model the internal optical parameters (focal length, principal point) of the virtual camera through a standardized 3×3 intrinsic parameter matrix, and to model the external pose parameters (rotation, translation) of the virtual camera through a 4×4 homogeneous extrinsic parameter matrix. The two types of matrices together constitute a complete camera projection model, realizing accurate projection of the 3D UE scene to the 2D original image sequence, and providing accurate camera parameter input for subsequent Gaussian sputtering scene reconstruction.

[0101] Extract the projection matrix of the original image sequence, determine the intrinsic and extrinsic parameter matrices of the original image sequence, and obtain a parameter set. In one implementation, the projection matrix is ​​a 3x4 matrix directly obtained from the rendering engine, and the intrinsic parameter matrix includes the focal length and principal point coordinates, such as... The extrinsic parameter matrix is ​​then determined by the rotation matrix. and displacement vector Composition, through decomposition of the projection matrix To determine this, for example, when processing a sequence of virtual city images, functions from the OpenCV library are used to extract these matrices from each frame. The intrinsic parameters reflect the camera's internal properties, such as distortion correction, while the extrinsic parameters describe the camera's pose in the world coordinate system, thus forming a set of parameters that supports precise calculations for image alignment.

[0102] Step S130: Use a transformation operator to perform a spatial transformation on the extrinsic parameter matrix in the parameter set to obtain the aligned camera extrinsic parameter matrix.

[0103] The aligned camera extrinsic matrix is ​​obtained using the following formula: (3) In formula (3), This is the aligned camera extrinsic matrix, which is unitless, derived from spatial transformation operations, and is a 4×4 matrix adapted to the right-handed coordinate system. This is a transformation operator, without units, used to perform coordinate system mapping transformations, realizing the transformation from a left-handed coordinate system to a right-handed coordinate system; The original camera extrinsic matrix is ​​unitless and is derived from the extraction result of formula (2); This is a coordinate system transformation matrix, unitless, a preset 4×4 matrix, fixed in size. , used to achieve y-axis reversal to complete coordinate system transformation. The control logic of formula (3) takes the UE's native left-handed coordinate system extrinsic matrix as input, and performs rigid coordinate transformation by fusing the preset y-axis reversal transformation matrix through the transformation operator. This maps the camera pose in the UE's left-handed coordinate system to the aligned extrinsic matrix in the computer vision standard right-handed coordinate system, solving the coordinate system incompatibility problem between the UE engine and the reconstruction algorithm, and providing a unified and standard coordinate reference for subsequent Gaussian sputtering scene reconstruction.

[0104] A transformation operator is used to perform a spatial transformation on the extrinsic parameter matrices in the parameter set to obtain an aligned camera extrinsic parameter matrix. For example, the transformation operator can be an affine transformation matrix, which optimizes the rotation and translation components of the extrinsic parameter matrix using the least squares method to align it to the reference coordinate system. In virtual reality roaming applications, if the original extrinsic parameter matrix has deviations, applying a transformation operator such as a rotation transformation yields an aligned matrix, thereby ensuring the consistency of multi-view images.

[0105] Step S140: Perform geometric correction on the original image sequence based on the aligned camera extrinsic parameter matrix to obtain the aligned image sequence and camera parameters.

[0106] The pixel values ​​of the aligned image sequence are obtained using the following formula: (4) In formula (4), For the aligned image sequence in pixel coordinates The pixel value at that location is unitless, derived from the geometric correction result, and its range is [value range missing]. (Grayscale value) or (RGB color values); These are image pixel coordinates, unitless, and their values ​​range from positive integers corresponding to the image resolution (e.g., ...). , , Image width, (Image height); The original image sequence is unitless and originates from the rendering result of step S110; it is a pixel value matrix. The aligned camera extrinsic matrix is ​​unitless and derived from the calculation result of formula (3); It is a projection inverse operator, unitless, used to convert pixel coordinates to world space coordinates; The image width and height are in pixels (px), derived from the resolution parameters of the original image sequence, and their values ​​are positive integers. The control logic of formula (4) is based on the geometric reprojection correction of the aligned camera extrinsic parameters. Starting from the pixel coordinates on the target aligned image, the projection inverse operator is used to reverse map to the world coordinate system space points. Then, combined with the original image sequence and the aligned camera extrinsic parameter matrix, spatial consistency resampling is completed to obtain the aligned pixel values ​​after geometric correction. The whole process uses a unified right-handed coordinate system as the reference to eliminate the image geometric distortion and offset caused by differences in camera pose and coordinate system, so that the original image sequence of multiple views is strictly aligned in space, providing high-quality image input with geometric consistency and regular viewpoint for subsequent Gaussian sputtering scene reconstruction.

[0107] The original image sequence is geometrically corrected based on the aligned camera extrinsic matrix to obtain the aligned image sequence and camera parameters. In one implementation, the geometric correction involves homography transformation, mapping image pixels to a uniform plane. For example, the perspective transformation matrix is ​​calculated using the aligned extrinsic matrix, and a warp operation is performed on each frame to eliminate distortion and misalignment, thereby obtaining the corrected sequence and final camera parameters, enhancing the immersive experience of virtual reality.

[0108] Preferably, the method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering provided in this embodiment includes step S200: Step S210: The aligned image sequence is processed by a median filter operator to remove pixel noise, thereby obtaining a denoised image sequence.

[0109] The pixel values ​​of the denoised image sequence are obtained using the following formula: (5) In formula (5), The denoised image sequence in pixel coordinates The pixel value at that location is unitless and comes from median filtering; its value range is consistent with the aligned image. This is a median filtering operator with no unit, used to remove pixel noise, and employs a 3×3 filtering window. For the aligned image sequence in pixel coordinates The pixel value at the location is unitless and comes from the calculation result of formula (4); The half-width of the filter window is unitless and takes a value of 1 (corresponding to a 3×3 window), which is a fixed empirical value. This is the filter window offset, unitless, and its value is... The control logic of formula (5) is based on the target pixels of the aligned image. Centered on the image, all pixel values ​​within a 3×3 neighborhood window are taken. The median of the pixels within the window is then used as the denoised pixel value by a median filter operator. This effectively removes pixel-level interference such as salt-and-pepper noise and impulse noise while preserving the image edge details and scene structure, providing high-quality, noise-free image input for subsequent Gaussian sputtering reconstruction.

[0110] In virtual reality building simulation systems, the aligned image sequence is first processed by a median filter operator to remove pixel noise. Specifically, the median filter operator is a non-linear filtering method that sorts the neighborhood of each pixel in the image and selects the median value to replace the original pixel, thereby effectively removing salt-and-pepper noise or random interference. For example, when processing a scan sequence of the interior of a virtual building, the filter window size is set to 3x3, and the median value is calculated pixel by pixel in each frame of the aligned image sequence to obtain a denoised image sequence, which helps improve the accuracy of subsequent feature extraction.

[0111] Step S220: Extract grayscale gradient features from the denoised image sequence, and use the Laplacian operator to calculate edge contrast to obtain a sharpened image sequence.

[0112] The pixel values ​​of a sharpened image sequence are obtained using the following formula: (6) In formula (6), For the sharpened image sequence in pixel coordinates The pixel value at that location is unitless and originates from edge enhancement processing; its value range is [value range missing]. or ; The image sequence after denoising is unitless and is derived from the calculation result of formula (5); This is the sharpening factor, unitless, derived from empirical configuration, and its value range is [range missing]. , used to control the sharpening intensity; The Laplacian operator, dimensionless, is used to calculate the second derivative of the image grayscale gradient and extract edge features. The operator template is... The control logic of formula (6) is based on the denoised image. It uses the Laplacian operator to calculate the second derivative of the image grayscale to extract edge features, and then uses the sharpening coefficient λ to control the enhancement intensity. The second derivative component is subtracted from the original pixel value to enhance the edge contour and highlight the texture details. At the same time, it retains the original information of the smooth area of ​​the image, providing a high-contrast and high-detail image input for subsequent Gaussian sputtering reconstruction.

[0113] When extracting grayscale gradient features from denoised image sequences, the Laplacian operator is used to calculate edge contrast. The Laplacian operator is a second-order differential operator used to detect edges and details in images, highlighting areas with dramatic grayscale changes through convolution operations. Specifically, in virtual reality systems, the denoised image is first converted to a grayscale image, then a Laplacian template such as [0, 1, 0; 1, -4, 1; 0, 1, 0] is applied for convolution to calculate the gradient value of each pixel. Contrast is then enhanced through thresholding to obtain a sharpened image sequence. For example, when simulating building outlines, this process can clearly highlight wall edges, improving the visual sharpness of the image and providing a more reliable foundation for depth processing.

[0114] Step S230: Simultaneously import the sharpened image sequence into the depth texture and normal vector, and perform back projection through the camera matrix to determine the spatial coordinate set.

[0115] The three-dimensional coordinates in the spatial coordinate set are obtained by the following formula: (7) In formula (7), These are three-dimensional coordinates in the set of spatial coordinates, in meters, derived from back projection calculations, and their values ​​range from the spatial coordinate interval of the virtual reality environment. These are three-dimensional coordinates in world space, in meters, and their values ​​range from the spatial coordinate interval of the virtual reality environment. The aligned camera extrinsic matrix is ​​unitless and derived from the calculation result of formula (3); is the inverse matrix of the camera intrinsic parameter matrix, which has no unit and is derived from formula (2) for finding the inverse of the intrinsic parameter matrix; These are pixel coordinates of the image, without units, derived from a sharpened image sequence. The depth value corresponding to the pixel is in meters (m), derived from depth texture data, and its value range is positive real numbers. The control logic of formula (7) starts with the pixel coordinates of the sharpened image sequence, combines the intrinsic parameter inverse matrix to convert the pixel into a camera coordinate system direction vector, transforms the direction vector to world space through the aligned extrinsic parameter matrix, and finally multiplies it by the pixel depth value to obtain the three-dimensional coordinates of the world space corresponding to the pixel. This formula realizes the accurate inverse projection from two-dimensional image pixels to three-dimensional world space, providing a complete set of spatial coordinates for the domestic reconstruction of UE scenes.

[0116] The sharpened image sequence is synchronously imported into the depth texture and normal vectors, and then back-projected using the camera matrix to determine the set of spatial coordinates. The camera matrix here refers to the combination of intrinsic and extrinsic parameters used to map 2D image points back to 3D space, while back-projection involves inverse transformation calculations. Specifically, in a virtual architectural environment, the depth texture provides the depth value for each pixel, and the normal vector represents the surface direction. Through the inverse operation of the camera matrix (similar to the projection matrix), the pixels of each frame are projected into the world coordinate system using a formula, forming a set of coordinates. For example, for a series of interior views of a building, back-projection can generate a set of points (x, y, z), supporting subsequent 3D modeling.

[0117] Step S240: Construct geometric topological relationships based on the set of spatial coordinates. If the projection residual is lower than the preset iteration threshold, generate the initial point cloud.

[0118] The projection residual and the initial point cloud are obtained using the following formulas: (8) In formula (8), This is the projection residual, expressed in pixels (px), derived from the calculation of the deviation between spatial coordinates and pixel coordinates, with a value range of [value range missing]. ; This is a projection operator, unitless, used to convert world space coordinates into image pixel coordinates; The set of spatial coordinates, in meters, is derived from the calculation results of formula (7); These are pixel coordinates of the image, without units, derived from a sharpened image sequence. It is a 2-norm (Euclidean distance) operator, without units, used to calculate the deviation between projected coordinates and actual pixel coordinates; The preset iteration threshold, in pixels (px), is derived from empirical configuration and has a range of values. ; The initial point cloud is unitless and is a set of spatial coordinates, with each point containing three-dimensional coordinates and corresponding color information. The control logic of formula (8) is to first reproject the spatial coordinates back to the image plane using the projection operator, calculate the Euclidean distance between the projected coordinates and the original pixel coordinates using the 2-norm to obtain the projection residual, and quantify the geometric accuracy of the spatial coordinates; then, using a preset iteration threshold as the filtering condition, only spatial coordinates with residuals below the threshold are retained to generate a noise-free, high-precision initial point cloud, providing a precise three-dimensional geometric reference for subsequent Gaussian sputtering scene reconstruction.

[0119] Geometric topological relationships are constructed based on a set of spatial coordinates. If the projection residual is lower than a preset iteration threshold, an initial point cloud is generated. Geometric topological relationships refer to the mesh structure formed by connections between points. The projection residual is a measure of reprojection error, which is minimized through iterative optimization, such as the least squares method. Specifically, in the domestic migration and reconstruction system, the Delaunay triangulation algorithm is first used to connect coordinate points to form a topological mesh. Then, the error of each point projected onto the image plane is calculated. If the average residual is less than a threshold of 0.5 pixels, it is considered converged, and an initial point cloud is generated. For example, in virtual building reconstruction, this step ensures the geometric consistency of the point cloud and avoids distortion.

[0120] Step S250: Match the sampling frequency of the initial point cloud with the sharpened image sequence, and perform dense reconstruction by verifying the mapping accuracy to obtain the preprocessed image sequence and the initial point cloud.

[0121] The sampling frequency of the initial point cloud is obtained using the following formula: (9) In formula (9), The sampling frequency matching coefficient is a dimensionless coefficient derived from the comparison of the sampling frequencies of the point cloud and the image, and its value range is [value range missing]. A value closer to 1 indicates a higher degree of matching; The sampling frequency of the initial point cloud is expressed in points per square meter (m²), derived from the statistics of the initial point cloud, and its value range is a positive real number. The sampling frequency for sharpening the image sequence, in pixels per square meter (m²), is calculated based on image resolution and scene size, and its value range is a positive real number. This is a dense reconstruction operator with no unit, used to perform point cloud densification processing and increase point cloud density. The initial point cloud has no units and is derived from the calculation results of formula (8); The image sequence is sharpened, and has no unit; it is derived from the calculation result of formula (6). The initial point cloud after densification is unitless and is the final initial point cloud used in subsequent steps. The control logic of formula (9) is to first calculate the sampling frequency matching coefficient by the ratio of the initial point cloud sampling frequency to the sharpened image sampling frequency, and quantify the consistency of the sampling density between the point cloud and the image; then, with the matching coefficient as a constraint, the initial point cloud is densified by the dense reconstruction operator, so that the point cloud density is precisely aligned with the image sampling density, generating a dense initial point cloud with high integrity and high consistency, providing the optimal geometric input for subsequent Gaussian sputtering scene reconstruction.

[0122] The initial point cloud and the sharpened image sequence are sampled at the same frequency. Dense reconstruction is then performed by verifying the mapping accuracy, resulting in a preprocessed image sequence and the initial point cloud. Sampling frequency matching involves synchronizing the point cloud density with the image resolution, while mapping accuracy verification is performed by comparing the deviations in the projected coordinates. Specifically, in virtual reality applications, the point cloud is first downsampled to match the image frame rate. Then, algorithms such as Bundle Adjustment are used to optimize the mapping. If the accuracy deviation is less than a threshold, gaps are filled to perform dense reconstruction. For example, in architectural simulations, this process generates a high-density point cloud and an optimized image sequence, improving the model's detail. Through this process, high-quality 3D reconstruction results are obtained, enhancing the immersive experience of virtual reality.

[0123] Furthermore, the UE scene localization migration and reconstruction method based on Gaussian sputtering provided in this embodiment includes step S300 as follows: Step S310: Determine the covariance matrix based on the geometric topological relationship between the preprocessed image sequence and the initial point cloud, and obtain the ellipsoidal morphological features corresponding to the covariance matrix.

[0124] The covariance matrix is ​​determined using the following formula: (10) In formula (10), It is a covariance matrix, without units, derived from the geometric topological relationship calculation of point cloud, and is a 3×3 symmetric matrix used to control the shape of the ellipsoid; The initial number of points in the point cloud is a unitless number derived from the initial point cloud statistics and its value ranges from positive integers. For the initial point cloud, the first The three-dimensional coordinates of each point, in meters, are derived from formula (9) for dense point cloud; These are the center coordinates of the initial point cloud, in meters, derived from the calculation of the average point cloud coordinates. ; The matrix transpose operator is unitless. The control logic of formula (10) takes the dense initial point cloud as input, first calculates the center coordinates of the point cloud as the distribution benchmark, and then performs the "coordinate-center" centering process on each point. By summing the outer product of the centering vector and its transpose and taking the average, a 3×3 covariance matrix representing the three-dimensional spatial distribution of the point cloud is obtained. The eigenvalues ​​of this covariance matrix correspond to the lengths of the three semi-axis of the ellipsoid, and the eigenvectors correspond to the directions of the three principal axes of the ellipsoid, thereby accurately determining the ellipsoidal morphological features corresponding to the point cloud and providing core geometric parameters for the initialization of the Gaussian ellipsoid in subsequent Gaussian sputtering.

[0125] In virtual reality building simulation systems, the covariance matrix is ​​determined based on the geometric and topological relationship between the preprocessed image sequence and the initial point cloud, thereby obtaining the ellipsoidal morphological features corresponding to the covariance matrix. For example, when processing 3D reconstruction of the interior of a building, the spatial distribution of each point in the point cloud is first analyzed, and the covariance matrix is ​​constructed by calculating the statistical characteristics of the point set. Specifically, the covariance matrix is ​​a symmetric positive definite matrix used to describe the dispersion and correlation of point cloud data. The matrix elements represent the covariance values ​​between variables. By centering the point cloud coordinates (i.e., subtracting the mean vector), the product of the transpose matrix and itself is calculated and divided by the number of points minus one, resulting in the covariance matrix, which reflects the ellipsoidal distribution characteristics of the point cloud. For example, when simulating the structure of a high-rise building, the eigenvectors of this covariance matrix correspond to the principal axis direction of the ellipsoid, and the eigenvalues ​​represent the axis lengths, helping to identify the overall shape tendency of the point cloud.

[0126] Step S320: Spatial constraints are applied to the initial point cloud using ellipsoidal morphological features, and the local illumination intensity is fitted using spherical harmonic functions to determine the spherical harmonic components, thereby obtaining a color and shadow distribution model.

[0127] Local illumination intensity is obtained using the following formula: (11) In formula (11), The local illumination intensity is expressed in cd / m² (candela per square meter), derived from the fitting results of the spherical harmonic function, and its value range is positive real numbers. These are the azimuth and polar angles, measured in rad (radians), derived from the direction of the normal to the point cloud surface. , ; This is the highest order of the spherical harmonic function, which is dimensionless and derived from empirical configuration, taking a value of 2 or 3 (to balance fitting accuracy and computational cost). For the first Rank The sub-spherical harmonic function is a unitless, predefined spherical function used to express the directional characteristics of light. These are the spherical harmonic component coefficients, which are dimensionless and derived from the light intensity fitting calculation. Their values ​​are real numbers, representing the weights of each order of spherical harmonic function. Let be the order of the spherical harmonic function. The control logic of formula (11) is based on the polar angle corresponding to the surface normal of the point cloud. Azimuth As input, pass through the highest order Constraining the fitting accuracy, using spherical harmonic functions of each order With corresponding spherical harmonic component coefficients By weighted superposition, the spatial distribution of local illumination intensity is accurately fitted, and a color and shadow distribution model that can characterize complex ambient light is constructed. The model uses ellipsoidal morphological features as spatial constraints to ensure that the illumination fitting is strictly aligned with the scene geometry, providing accurate lighting parameters for subsequent Gaussian sputtering color and illumination restoration.

[0128] The initial point cloud is spatially constrained using the ellipsoidal morphological features. Spherical harmonic functions are then used to fit local illumination intensity to determine spherical harmonic components, thus obtaining a color and shadow distribution model. Specifically, a spherical harmonic function is an orthogonal basis function based on spherical coordinates used to represent illumination distribution. Low-order coefficients capture global illumination, while high-order coefficients describe local variations. In the domestic migration and reconstruction system, constraint boundaries are first defined based on ellipsoidal features, such as limiting point cloud deformation to the ellipsoidal boundary. Then, illumination intensity in the image sequence is sampled, and the least squares method is applied to fit the spherical harmonic coefficients. For example, in architectural lighting simulation, illumination data for the skylight area is fitted to obtain spherical harmonic components, such as zero-order representing ambient light and first-order representing directional light. This constructs a color and shadow distribution model to describe color gradation and shadow distribution on the surface. This process ensures that the point cloud maintains geometric consistency in space while incorporating realistic lighting effects.

[0129] Step S330: Establish a color mapping function based on the color and light distribution model, map the spherical harmonic components to the surface corresponding to the ellipsoidal morphological features, and obtain the initial three-dimensional model surface texture with light and shadow features.

[0130] The color value at a corresponding location in the point cloud is obtained using the following formula: (12) In formula (12), This represents the color value at the corresponding location in the point cloud. It has no unit and originates from color mapping operations, with a value range of [value range missing]. (RGB color values); This is a color mapping function with no unit, used to blend light intensity with image color to generate point cloud surface color; The local illumination intensity is expressed in cd / m² and is derived from the fitting results of formula (11). The pixel color values ​​of the image sequence are for sharpening, and have no unit. They are derived from the calculation results of formula (6). This is the fusion coefficient, which is unitless, derived from empirical configuration, and its value range is [value range missing]. This is used to balance the proportion of lighting and the original image colors; The coordinates of the point cloud are in three dimensions, in meters. The corresponding pixel coordinate is unitless. The control logic of formula (12) uses the three-dimensional coordinates of the point cloud and the corresponding pixel coordinates as a spatial bridge. It uses a color mapping function to weight and fuse the local illumination intensity of the spherically harmonic fitting with the original color of the sharpened image, and uses the fusion coefficient as the basis for the result. By balancing the proportion of lighting effects and original textures, the spherical harmonic components are accurately mapped to the point cloud surface corresponding to the ellipsoidal morphological features, generating point cloud surface colors that combine realistic lighting and original textures, providing high-fidelity surface textures for the initial 3D model.

[0131] A color mapping function is established based on the color and light distribution model to map the spherical harmonic components to the surface corresponding to the ellipsoidal morphological features, thereby obtaining an initial 3D model surface texture with light and shadow characteristics. Specifically, the color mapping function is a conversion mechanism from illumination components to surface colors. It calculates the RGB value of each surface point by linearly combining spherical harmonic basis functions and coefficients. In a virtual architectural environment, the ellipsoidal surface is parameterized, such as by using UV coordinates, and then the spherical harmonic components are projected onto these coordinates. For example, when simulating an interior wall, the mapping function considers the normal direction and the light source angle to generate a texture map that displays highlight and shadow areas, thus making the surface of the color and light distribution model present a realistic visual effect.

[0132] Step S340: Fine-tune the ellipsoid shape by performing surface texture adjustment on the initial three-dimensional model, match the eigenvalues ​​of the covariance matrix with the preset geometric constraints, and obtain the initial three-dimensional model.

[0133] The initial 3D model is derived using the following formula: (13) In formula (13), This is an initial 3D model, without units, derived from ellipsoidal shape fitting and color mapping results, containing geometric shape and surface texture; This is a fitting operator with no units, used to fit a point cloud into an ellipsoidal set based on Gaussian sputtering, and then combine it with color information to generate a 3D model. The initial point cloud is densified and has no units; it is derived from the calculation results of formula (9). The covariance matrix is ​​unitless and is derived from the calculation results of formula (10); The color value of the point cloud surface is unitless and is derived from the calculation result of formula (12); Covariance matrix The eigenvalues ​​of are unitless and derived from the eigenvalue decomposition of the covariance matrix. They are used to characterize the proportions of the major, middle, and minor axes of the ellipsoid. The preset geometric constraint threshold is dimensionless, derived from empirical configuration, and its value range is positive real numbers. It is used to limit the rationality of the ellipsoid shape. The control logic of formula (13) is based on the density of the initial point cloud as the geometric basis. The point cloud is mapped to a set of 3D Gaussian ellipsoids through fitting operators. The shape, direction and size of the ellipsoids are precisely controlled by the eigenvalues / eigenvectors of the covariance matrix. At the same time, the color values ​​of the point cloud surface are integrated to give the model a realistic texture. The rationality of the ellipsoid shape is then verified by the preset geometric constraint threshold, abnormal ellipsoids are eliminated, and finally an initial three-dimensional model with both accurate geometric structure and high-fidelity surface texture is generated, providing the optimal initial input for subsequent Gaussian optimization and scene rendering.

[0134] The initial 3D model undergoes fine-tuning of its ellipsoidal shape by adjusting the surface texture, matching the eigenvalues ​​of the covariance matrix with preset geometric constraints to obtain the final 3D model. Specifically, this adjustment involves iteratively optimizing the surface shape to conform to the scale represented by the matrix eigenvalues. For example, in architectural models, the ellipsoidal axis length is progressively scaled to match constraints such as upper curvature limits, thereby optimizing texture fit. This process enhances the realism and stability of the initial 3D model.

[0135] Preferably, the method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering provided in this embodiment includes step S400: Step S410: Obtain the ellipsoidal morphological features and spherical harmonic components contained in the initial 3D model, and perform rasterization processing through a preset viewpoint transformation matrix to obtain the rendered image.

[0136] The rendered image is obtained using the following formula: (14) In formula (14), For rendering, it has no units, is derived from the rasterization process, and is a pixel value matrix, consistent with the resolution of the original virtual reality environment image. It is a rasterization operator with no units, used to convert 3D models into 2D rendered images; The initial three-dimensional model has no units and is derived from the calculation results of formula (13); The preset viewpoint transformation matrix is ​​a unitless 4×4 matrix derived from the renderer configuration and used to determine the rendering perspective. The control logic of formula (14) takes the Gaussian ellipsoid set of the initial 3D model as input, converts the 3D coordinates of the world space into screen space coordinates through the preset viewpoint transformation matrix, and then performs view frustum clipping, ellipsoidal projection, depth sorting and pixel fusion by the rasterization operator to convert the 3D model into a 2D rendering screen with the same resolution as the original image. This formula realizes efficient visualization of the 3D scene and provides gradient feedback for differentiable rendering for subsequent model optimization. It solves the technical pain points of low rendering efficiency, high cost of viewpoint adjustment and lack of optimization loop in traditional rendering. It is the core rendering technology for the localization migration and reconstruction of UE scene based on Gaussian sputtering.

[0137] In a virtual reality architectural simulation system, the ellipsoidal morphological features and spherical harmonic components of the initial 3D model are first acquired. The ellipsoidal morphological features describe the shape distribution of the point cloud through a covariance matrix, while the spherical harmonic components represent the spherical functions of lighting and color, thus providing basic data for subsequent rendering. For example, when reconstructing the interior of a high-rise building, rasterization is performed using a preset viewpoint transformation matrix. This viewpoint transformation matrix is ​​a 4x4 transformation matrix used to project 3D coordinates onto a 2D screen space. The specific process includes multiplying the vertex coordinates of the initial 3D model by the viewpoint matrix to obtain normalized device coordinates, then performing perspective division and viewport transformation to finally generate the rendered image. For example, the ellipsoidal points of the building lobby model are rasterized and filled with pixels to simulate the image output from the perspective, thereby obtaining an image comparable to the original virtual reality environment image.

[0138] Step S420: Calculate the pixel residual and determine the mean square error of the current iteration cycle based on the pixel-by-pixel comparison between the rendered image and the original virtual reality environment image.

[0139] The mean square error of the current iteration period is obtained by the following formula: (15) In formula (15), This represents the mean squared error for the current iteration period, without units, derived from a pixel-by-pixel comparison between the rendered image and the original image, with a value range of [value range missing]. The smaller the value, the smaller the error; These are the image width and height, in pixels (px), derived from the image resolution parameters. To render the image in pixel coordinates The pixel value at the location is unitless and is derived from the calculation result of formula (14); The original virtual reality environment image in pixel coordinates The pixel value at that location is unitless and originates from the original image sequence in step S100. The square of the 2-norm is dimensionless and is used to calculate the squared error between two pixel values. The control logic of formula (15) is based on the one-to-one correspondence between the pixel level of the rendered image and the original image. The residual of each pixel is calculated by the square of the 2-norm, and then the residuals of all pixels are summed and averaged to obtain the mean square error of the current iteration cycle. This error quantifies the reconstruction accuracy of the 3D model and serves as the core loss index for model optimization, driving the iterative optimization of the Gaussian parameters to achieve accurate alignment between the 3D model and the original scene.

[0140] Based on a pixel-by-pixel comparison between the rendered image and the original virtual reality environment image, the pixel residuals are calculated and the mean square error of the current iteration cycle is determined. The pixel residual refers to the difference in the RGB values ​​of the corresponding pixels, and the mean square error is obtained by averaging the squares of all residuals. For example, in a simulated architectural lighting scene, the rendered wall shadows are compared with the original image, and the residuals are calculated, such as the difference in the red channel (0.1), green (0.05), and blue (0.08). Then, the sum of the squares is divided by the number of pixels to obtain an error value of 0.02, which is used to quantify the accuracy of the model. This process ensures that the iterative optimization targets the real differences, thereby improving the realism of the model.

[0141] Step S430: Calculate the gradient operator for the ellipsoidal morphology and spherical harmonic components using the backpropagation algorithm based on the mean square error, and obtain the iteration step size for numerical update.

[0142] The iteration step size for numerical updates is derived using the following formula: (16) In formula (16), This is the iteration step size for numerical updates, which is unitless and comes from the backpropagation algorithm. It is used to correct the parameters of the 3D model. The learning rate is unitless, derived from model optimization configuration, and its value ranges from [value missing]. This is used to control the iteration update speed; This is the gradient of the mean squared error with respect to the model parameters. It is dimensionless, originates from the backpropagation algorithm, and points in the direction where the error decreases. This is a set of model parameters, unitless, containing ellipsoidal morphological features (covariance matrix). ), spherical harmonic components ( ), and the spatial position of the three-dimensional model. The control logic of formula (16) is based on the mean square error. To optimize the objective, the backpropagation algorithm is used to calculate... The gradient of all model parameters with respect to the learning rate By controlling the update magnitude, parameter iteration step size is generated along the gradient in the opposite direction (the direction of error reduction) to correct model parameters such as ellipsoid shape, spherical harmonic components, and spatial position, thereby driving continuous improvement in the accuracy of the 3D model and realizing an end-to-end optimization closed loop of "error-gradient-update".

[0143] The backpropagation algorithm is used to calculate the gradient operator for the ellipsoidal morphological features and the spherical harmonic components based on the mean square error, thereby obtaining the iteration step size for numerical updates. The backpropagation algorithm is a chain-like differentiation method that propagates from the output error to the input layer, calculating the gradient of each parameter. For example, in architectural model optimization, starting from the mean square error, the partial derivative of the ellipsoidal covariance is calculated in reverse, such as the derivative with respect to the matrix elements, which is approximately 0.001. Then, combined with a learning rate of 0.01, a step size of 0.00001 is determined for parameter adjustment. This backpropagation algorithm minimizes the error through gradient descent, achieving efficient iteration.

[0144] Step S440: The spatial position, covariance matrix and color and light distribution model of the initial 3D model are numerically corrected using an iterative step size to obtain the corrected 3D model.

[0145] The spatial position of the initial 3D model, the covariance matrix, and the color and light distribution model are numerically corrected using the iteration step size to obtain a corrected 3D model. For example, when simulating skylight illumination, the position coordinates are corrected by shifting the x-axis by 0.5 units, the diagonal elements of the covariance matrix are updated by increasing by 0.1, and the spherical harmonic coefficients are adjusted to match the shadow gradient, thereby generating a more accurate model version.

[0146] Step S450: If the optimization error generated by correcting the 3D model is lower than the preset threshold, then stop the iteration process and obtain the optimized 3D model.

[0147] The correction and optimization of the 3D model are derived using the following formulas: (17) In formula (17), This is a correction of the 3D model, without units, derived from the iterative correction results of the model parameters, and is the model after the initial 3D model has been updated by step size; The initial three-dimensional model has no units and is derived from the calculation results of formula (13); The iteration step size is unitless and is derived from the calculation result of formula (16); To optimize the 3D model, it is unitless, derived from the iteration stop determination result, and is the final optimized model that meets the error requirements; The mean square error of the current iteration is dimensionless and is derived from the calculation result of formula (15); This is a preset optimization error threshold, without units, derived from empirical configuration, and its value range is [value range missing]. If the error is lower than the optimization error threshold, the iteration stops. The control logic of formula (17) is to first generate a corrected three-dimensional model by superimposing the initial three-dimensional model with the iteration step size, and complete the single-round parameter update; then, based on the mean square error of the current iteration... Compared with the preset optimization error threshold As a basis for judgment, if If the value is below the threshold, the iteration stops and an optimized 3D model is output; otherwise, the corrected model is used as the initial model for the next iteration, and the entire process of rendering, error calculation, and gradient update is repeated until the model converges. Finally, an optimized 3D model that meets the accuracy requirements is output, realizing high-precision domestic reconstruction of UE scenes.

[0148] If the optimization error generated by the corrected 3D model is lower than a preset threshold such as 0.001, the iteration process is stopped, and an optimized 3D model is obtained. This optimization ensures the stability and visual realism of the building simulation.

[0149] Furthermore, the method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering provided in this embodiment includes step S500: Step S510: Obtain the center coordinates and covariance matrix of the optimized 3D model, and transform the ellipsoidal morphological features of the optimized 3D model through the attribute mapping mechanism to obtain the standard format of the point cloud.

[0150] The standard format of point clouds is derived using the following formula: (18) In formula (18), It is a standard point cloud format, without units, derived from the result of attribute mapping transformation, and adopts the PCL standard format (such as .pcd), containing attributes such as point cloud coordinates, color, and normal vector; It is an attribute mapping mechanism with no units, used to convert the ellipsoidal morphological features of an optimized 3D model into standard point cloud attributes; To optimize the 3D model, no units are used; the results are derived from the calculation of formula (17). The parameters are Gaussian sputtering parameters, which are unitless and include Gaussian kernel size, sputtering intensity, etc., derived from the Gaussian sputtering algorithm configuration. The control logic of formula (18) is based on the geometric and textural foundation of the optimized 3D model's Gaussian ellipsoid set. Through the attribute mapping mechanism, it integrates the configuration parameters of the Gaussian sputtering algorithm to accurately convert the ellipsoidal morphological features, surface color, and lighting information of the optimized 3D model into a point cloud data format conforming to the PCL standard. This formula realizes the lossless mapping from the optimized 3D model to a general point cloud, providing a standardized data interface for subsequent domestic reconstruction algorithms, visualization rendering, and engineering applications.

[0151] In a virtual reality building simulation system, the center coordinates and covariance matrix of the optimized 3D model are first obtained. These parameters describe the position and shape distribution of the ellipsoid in the optimized 3D model. For example, the center coordinates represent the geometric center point of the ellipsoid, and the covariance matrix quantifies the extension direction and scale of the point cloud through its eigenvalues ​​and eigenvectors. For example, the ellipsoidal morphological features of the optimized 3D model can be transformed through an attribute mapping mechanism to obtain a standard point cloud format. The attribute mapping mechanism is a data transformation process that maps the morphological parameters of the ellipsoid, such as radius and rotation angle, to a standard point cloud structure. Specifically, the attribute mapping mechanism first extracts the geometric attributes of the ellipsoid, and then applies a linear transformation to convert these attributes into vertex coordinates and normal vectors of the point cloud. For example, when processing a building exterior wall model, the center coordinates, such as (0, 0, 0), and the diagonal elements of the covariance matrix, such as 1.0, 2.0, and 3.0, are read from the optimized 3D model. The corresponding sampling points of the point cloud are calculated through a mapping function. The sampling process involves uniformly extracting a set of points from the ellipsoid surface and assigning attribute values ​​to each point, thereby generating a standard point cloud format compatible with multiple rendering engines. This standard point cloud format typically includes position, color, and normal vector fields to ensure data consistency and portability.

[0152] Step S520: Perform spatial index construction according to the point cloud standard format, and load the point cloud standard format into the spatial rendering engine using the memory alignment mechanism.

[0153] Spatial index construction is performed according to the point cloud standard format. The point cloud standard format is then loaded into the spatial rendering engine using a memory alignment mechanism. Spatial index construction refers to creating data structures such as octrees or KD trees to accelerate spatial queries. The memory alignment mechanism optimizes access efficiency by adjusting the layout of data in memory. For example, in a building interior reconstruction scene, when constructing an octree for the point cloud standard format, the point set is recursively divided into child nodes. Each node stores the bounding box and point index. At the same time, memory alignment is applied, such as padding the structure to a 64-byte boundary, thereby reducing cache misses when loading into the spatial rendering engine. The spatial rendering engine is a software framework dedicated to 3D graphics processing, responsible for managing rendering resources and executing the graphics pipeline.

[0154] Step S530: In the spatial rendering engine, perform tile division according to the viewpoint transformation matrix to determine the screen space area corresponding to the tile division.

[0155] The screen tiles are derived using the following formula: (19) In formula (19), For the first Line number Each screen tile, without units, originates from the tile partitioning result and represents a rectangular area of ​​screen space; A tile partitioning operator, without units, used to partition screen space based on the viewpoint transformation matrix and screen size; This is the viewpoint transformation matrix, which has no unit and is derived from the rendering engine configuration. These are the image width and height, in pixels (px), derived from the rendering resolution. The tile size is in pixels (px) and is based on empirical settings, with values ​​of 32×32 or 64×64. The tile's row and column index is unitless and takes values ​​in the range of positive integers, satisfying the following conditions: , The control logic of formula (19) uses the viewpoint transformation matrix as the coordinate transformation reference and divides the screen space into uniform rectangular grid areas according to the preset tile size based on the width and resolution of the image. This formula accurately generates the screen tile area corresponding to each row and column number through the tile division operator, realizes the spatial block management of 3D model rendering, and provides efficient spatial units for pipeline processing of parallel rendering, memory optimization and domestic reconstruction algorithms.

[0156] In the spatial rendering engine, tile partitioning is performed based on the viewpoint transformation matrix to determine the screen space region corresponding to the tile partitioning. The viewpoint transformation matrix is ​​a 4x4 matrix used to convert world coordinates to screen coordinates. Tile partitioning divides the screen into small regions for parallel processing. For example, after applying the viewpoint transformation matrix to the building lobby model, the screen is divided into 16x16 pixel tiles. Each tile corresponds to a screen space region with coordinates from (0, 0) to (16, 16), which facilitates subsequent rendering optimization.

[0157] Step S540: Use a depth sorting algorithm to arrange the rendering order of the point cloud standard format for the screen space region to obtain an ordered vertex stream.

[0158] The point cloud rendering order is determined by the following formula: (20) In formula (20), The point cloud rendering order is unitless, derived from the depth sorting algorithm, and is an ordered sequence of point cloud indices. This is a depth sorting algorithm operator, which is unitless and uses a back-to-fore depth sorting strategy. The point cloud is in standard format, without units, and is derived from the calculation results of formula (18); This is the viewpoint direction vector, which is unitless and derived from the viewpoint transformation matrix; it is a unit vector. The number of points in the standard point cloud, without units, derived from point cloud statistics, and the value range is a positive integer; For the first The depth value of each point, in meters, is derived from the dot product of the point cloud coordinates and the viewpoint direction. The sorting index operator is unitless and is used to return the point cloud indexes sorted by depth value. The control logic of formula (20) is based on the viewpoint direction vector extracted by the viewpoint transformation matrix to calculate the depth value of each point in the standard point cloud format; then, the depth sorting algorithm is used to arrange the point cloud indexes in order of depth value from far to near, generating an ordered vertex flow; this control logic ensures that during Gaussian sputtering rendering, distant points are written to the depth buffer first, and nearby points are rendered and covered later, ultimately achieving correct perspective occlusion and visual depth sorting.

[0159] For the screen space region, a depth sorting algorithm is used to arrange the rendering order of the point cloud standard format to obtain an ordered vertex flow. The depth sorting algorithm handles occlusion by calculating and sorting the distance from the point to the viewpoint. For example, when simulating architectural lighting, the depth value of each point in the point cloud is calculated, such as a distance of 5.0 units from the viewpoint, and then arranged from far to near to form an ordered vertex flow.

[0160] Step S550: Perform vertex shading and pixel blending on the ordered vertex stream through a dedicated rendering pipeline to obtain a preliminary rendering framework.

[0161] The initial rendering framework is derived using the following formula: (twenty one) In formula (21), This is a preliminary rendering framework, without units, derived from the processing results of a dedicated rendering pipeline, and includes rendering vertex streams, shading parameters, etc. It is a dedicated rendering pipeline operator with no unit, used to perform rendering processes such as vertex shading and pixel blending; The point cloud rendering order is unitless and derived from the calculation results of formula (20); The point cloud is in standard format, without units, and is derived from the calculation results of formula (18); These are dedicated rendering pipeline parameters, without units, and include vertex shader parameters, pixel shader parameters, etc., derived from the rendering engine configuration. The control logic of formula (21) uses the depth-sorted vertex stream as the execution order, the PCL standard format point cloud as the attribute basis, and through dedicated rendering pipeline operators, merges preset shaders and pipeline parameters to sequentially execute vertex shading, geometry processing, and pixel blending, converting the ordered Gaussian point cloud into a preliminary rendering framework containing a complete rendering state. This logic achieves lossless mapping from point cloud to rendering frame, providing high-fidelity rendering results for subsequent frame buffer fusion and final output.

[0162] A preliminary rendering framework is obtained by performing vertex shading and pixel blending on an ordered vertex stream through a dedicated rendering pipeline. This dedicated rendering pipeline is a customized graphics processing flow that includes a vertex shading stage for calculating lighting and a pixel blending stage for fusing colors. For example, in architectural models, shaders are applied to vertices to calculate shadows, and then pixels are blended to generate the framework image. Through this process, efficient model rendering is achieved.

[0163] Preferably, the method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering provided in this embodiment includes step S600 as follows: Step S610: Extract pixel color values ​​and vertex attributes from the initial rendering framework, perform linear interpolation on the pixel color values ​​using a preset blending factor, and determine the blended pixel sequence in conjunction with depth cache data.

[0164] The blended pixel color values ​​are obtained using the following formula: (twenty two) In formula (22), These are the blended pixel color values, without units, derived from alpha blending, and their range is [value range missing]. ; This is a blending factor, unitless, derived from rendering configuration, and its value range is [value range missing]. , This indicates that the foreground pixels are fully displayed. This indicates that the background pixels are fully displayed; This represents the color value of the foreground pixels, without units, derived from the pixel values ​​of the initial rendering framework, and its value range is... ; This is the background pixel color value, without units, derived from the background color configuration of the rendering engine, and its value range is... ; These are image pixel coordinates, without units. The control logic of formula (22) is based on pixel coordinates. Using the positioning benchmark, with a mixing factor As weights, linear interpolation is performed on foreground pixels (Gaussian sputtering rendering results) and background pixels (engine configuration colors); this logic, combined with depth cache data, ensures the correct fusion of transparent Gaussian point clouds and background colors, generating a final rendered image with natural transitions and no breaks, achieving high-fidelity visualization output of UE scenes.

[0165] Pixel color values ​​and vertex attributes are extracted from the initial rendering framework. For example, in a virtual reality building simulation system, the RGB color values ​​of each pixel in the framework image are first read, such as red channel intensity of 0.8, green of 0.6, and blue of 0.4. Simultaneously, vertex attributes, including position coordinates and texture coordinates, are extracted. This data comes from the output buffer of the rendering pipeline, providing the basic input for subsequent blending operations. A linear interpolation operation is performed on the pixel color values ​​using a preset blending factor, combined with depth cache data, to determine the blended pixel sequence. Specifically, the blending factor is a predefined weight value, such as 0.5, used to calculate the weighted average of two pixel color values. For example, when processing interior lighting simulations, the color values ​​of adjacent pixels are calculated using a formula such as the new color value equal to the factor multiplied by the first color value plus (1 - factor) multiplied by the second color value. Simultaneously, the depth value stored in the depth cache, such as 2.5 units, is referenced to determine the relationship between adjacent pixels, avoiding occlusion errors, thus generating a smooth blended pixel sequence. This blended pixel sequence is an ordered list of pixels, ensuring the continuity and realism of the rendering.

[0166] Step S620: Perform frame rate threshold monitoring on the mixed pixel sequence. If the current rendering frequency is lower than the preset frequency, perform downsampling processing on the mixed pixel sequence through a dynamic thinning algorithm to obtain a high-frequency rendering stream.

[0167] High-frequency rendering streams are derived using the following formula: (twenty three) In formula (23), This is a high-frequency rendering stream, without units, derived from downsampling processing results, and is a pixel sequence that meets frame rate requirements; This is a dynamic thinning algorithm operator with no unit, used to downsample mixed pixel sequences and improve rendering frame rate; The sequence of mixed pixels is unitless and is derived from the Alpha mixing result of formula (22). This is the preset rendering frequency, measured in fps (frames per second), derived from rendering performance requirements, and its value range is [range missing]. ; The current rendering frequency is expressed in fps, derived from frame rate monitoring results, and its value range is positive real numbers. The control logic of formula (23) is to first monitor the real-time rendering frame rate corresponding to the mixed pixel sequence, and compare the current rendering frequency with the preset rendering frequency. If the current rendering frequency is lower than the preset frequency, it is determined that the rendering performance is insufficient, and then the dynamic thinning algorithm is started to perform downsampling processing on the mixed pixel sequence. By reducing the number of pixels to be processed and the computational pressure within a unit frame, the overall rendering smoothness is improved, and finally a high-frequency rendering stream that meets the frame rate requirements is output. If the current rendering frequency has met the standard, the mixed pixel sequence is directly output as a high-frequency rendering stream without performing thinning downsampling, so as to preserve the integrity of the picture as much as possible while ensuring rendering performance.

[0168] Frame rate threshold monitoring is performed on the mixed pixel sequence. If the current rendering frequency is lower than a preset frequency, a dynamic thinning algorithm is used to downsample the mixed pixel sequence to obtain a high-frequency rendering stream. For example, in a dynamic browsing scene of an architectural model, if the current frame rate is monitored to be 20 frames per second, and it is lower than a preset 60 frames per second, a dynamic thinning algorithm is applied. This dynamic thinning algorithm adaptively removes redundant points according to the pixel density, for example, by extracting samples from every other pixel in the mixed pixel sequence, reducing the amount of data, thereby improving the rendering speed and obtaining a smoother high-frequency rendering stream.

[0169] Step S630: Extract geometric boundary features based on vertex coordinates in the high-frequency rendering stream, generate a set of collision voxels using a topology reconstruction algorithm, and obtain auxiliary collision mesh data.

[0170] The auxiliary collision mesh data is obtained using the following formula: (twenty four) In formula (24), The collision mesh data is unitless and derived from the topology reconstruction results; it serves as a mesh model for interactive collision detection. This is a topology reconstruction algorithm operator, which is unitless and used to generate collision voxels based on geometric boundary features. It represents geometric boundary features, is unitless, and originates from the vertex coordinate extraction of the high-frequency rendering stream; it is a set of boundary vertices. The voxel dimensions are in meters (m), derived from empirical configurations, and their range is [range missing]. The control logic of formula (24) is to extract vertex coordinates from the high-frequency rendering stream, filter and construct the geometric boundary feature set of the scene; use the geometric boundary as the shape constraint, combined with the preset voxel size, and discretize the continuous boundary into a set of regular voxels through the topology reconstruction algorithm; finally, these collision voxels are combined to generate auxiliary collision mesh data for interaction and physical detection, which provides a lightweight and directly usable collision interaction structure for domestic rendering engines while ensuring the consistency of scene outline.

[0171] Geometric boundary features are extracted from the vertex coordinates in the high-frequency rendering stream. A collision voxel set is then generated using a topology reconstruction algorithm to obtain auxiliary collision mesh data. Specifically, the geometric boundary features are determined by calculating the extreme values ​​of vertex coordinates, such as a maximum x-coordinate of 10.0. The topology reconstruction algorithm involves connecting adjacent vertices to form a mesh. For example, in simulating building collision detection, the coordinate points in the high-frequency rendering stream are recursively divided into voxels. Each voxel represents a cubic region, such as a side length of 1.0 unit, and its occupancy status is marked, thereby generating a collision voxel set. This set is further converted into auxiliary collision mesh data to support accurate calculations for interactive operations.

[0172] Step S640: Map the auxiliary collision mesh data to the interaction command response matrix, and match the interaction command response matrix with the rendering pipeline state through the compatibility interface to obtain the interactive compatibility rendering framework.

[0173] The interactive compatible rendering framework is derived from the following formula: (25) In formula (25), As an interactive and compatible rendering framework, it has no units and is derived from the results of compatibility interface matching, supporting real-time interaction and rendering; To ensure interface compatibility, this is a unitless matching operator used to match collision mesh data with rendering pipeline states, achieving interactive compatibility. It is a high-frequency rendering stream, without units, and is derived from the calculation results of formula (23); The collision mesh data is unitless and is derived from the calculation results of formula (24); The rendering pipeline state is unitless and originates from the current state of the dedicated rendering pipeline, including shader state, cache state, etc. The control logic of formula (25) is based on the high-frequency rendering stream as the visual basis and the auxiliary collision mesh as the interaction physical benchmark. Through the compatible interface matching operator, the collision mesh is mapped to the interaction command response matrix, and the real-time running state of the rendering pipeline is matched synchronously to establish a two-way binding between the interaction command and the rendering process. Finally, an interactive compatible rendering framework that supports real-time interaction, collision detection and smooth rendering is generated to realize full-function interactive adaptation after the localization migration of UE scene.

[0174] The auxiliary collision mesh data is mapped to an interaction command response matrix. A compatible interface is then used to match the interaction command response matrix with the rendering pipeline state, resulting in an interactively compatible rendering framework. For example, when a user interacts with a building model, such as touching it, the auxiliary collision mesh data is mapped to the interaction command response matrix, with each matrix element corresponding to a command response such as move or zoom. Then, a compatible interface, such as an API call, ensures that the interaction command response matrix is ​​synchronized with the rendering pipeline state, thereby achieving seamless interaction. This process improves the interactive efficiency of the rendering framework.

[0175] Furthermore, the UE scene localization migration and reconstruction method based on Gaussian sputtering provided in this embodiment includes step S700 as follows: Step S710: If rendering performance fluctuations are detected in the interactive compatible rendering framework, adjust the block sorting values ​​according to the geometric boundary features in the depth cache data.

[0176] The adjusted block sorting parameters are obtained using the following formula: (26) In formula (26), These are the adjusted block sorting parameters, which are unitless and derived from parameter adjustment results, including block size, sorting threshold, etc. This is a unitless operator for adjusting the sorting parameters in the chunks, used to adjust the sorting parameters based on rendering performance fluctuations. These are the original block sorting parameters, which have no unit and are derived from the initial rendering framework configuration. These are geometric boundary features, unitless, and derived from deep cached data extraction. This represents rendering performance fluctuation values, measured in frames per second (fps), derived from performance monitoring results, with a range of [range missing]. The larger the value, the more severe the fluctuation. The control logic of formula (26) is to monitor the rendering performance fluctuation value in real time. When a large fluctuation in frame rate is detected, the geometric boundary features are used as shape constraints. The block sorting parameter adjustment operator is used to adaptively quantize and adjust the original block sorting parameter. The calculation pressure is relieved or the rendering density is increased by dynamically shrinking / increasing the block size, adjusting the sorting threshold or optimizing the batch processing strategy. Finally, the block sorting parameter adapted to the current performance state is output to achieve stable and efficient rendering after the localization reconstruction of the UE scene.

[0177] When the interactive rendering framework runs in a virtual building simulation system, if rendering performance fluctuations are detected, such as the frame rate dropping from 60 frames per second to 30 frames per second, the geometric boundary features are first extracted from the depth cache data. For example, the coordinate range of the object's edge is determined by scanning depth values, thereby adjusting the chunk sorting values. Specifically, the chunk sorting value is a priority index based on boundary complexity. For example, when processing complex building models, the chunk values ​​for areas with dense boundary features are set to higher values, such as priority 5, to optimize the rendering order and avoid performance bottlenecks.

[0178] Step S720: Spatial clustering of vertex coordinates is performed using block sorting values ​​to obtain point cloud density information.

[0179] Point cloud density information and spatial clustering results are obtained using the following formulas: (27) In formula (27), This represents point cloud density information, with units of points / m², derived from spatial clustering and density statistics, and its value range is positive real numbers. This represents the number of points in a single spatial cluster. It is unitless, derived from clustering results statistics, and its value ranges from positive integers. This represents the area of ​​a single spatial cluster, expressed in square meters (㎡), derived from the spatial dimensions of the clustered region, and its value range is a positive real number. This is a spatial clustering operator with no unit, used to perform spatial clustering of vertex coordinates based on block sorting parameters; The point cloud is in standard format, without units, and is derived from the calculation results of formula (18); The adjusted block sorting parameters are unitless and derived from the calculation results of formula (26); The result is a spatial clustering result, without units, representing a set of clusters of point clouds divided according to their spatial distribution. The control logic of formula (27) uses the adjusted block sorting parameters as the spatial partitioning benchmark to perform spatial clustering on the PCL standard format point cloud, dividing the point cloud into multiple cluster sets according to the region; then, it counts the number of points and area within each cluster, calculates the point cloud density per unit area, and thus obtains the density distribution information of the entire scene. This logic realizes the dynamic quantization of point cloud density, providing core data support for subsequent density-based rendering optimization, LOD (Level of Detail) adjustment, and resource scheduling of domestic rendering engines.

[0180] The vertex coordinates are spatially clustered using the block sorting values ​​to obtain point cloud density information. In one implementation, spatial clustering involves grouping vertex coordinates. For example, in architectural simulation, the k-means algorithm is applied to the coordinate points, and the point cloud of high-priority blocks is clustered into high-density clusters based on the block sorting values, such as areas with more than 100 points per cubic meter. This allows the calculation of point cloud density information. This process is achieved by iteratively calculating the distance between the cluster center and the points, ensuring that the density information reflects the local complexity of the model and provides a data foundation for subsequent lighting and shadow processing.

[0181] Step S730: Align the light source vector with the surface normal during the topology reconstruction process based on the point cloud density information to handle the lighting consistency of the topology.

[0182] The topology after lighting and shadow consistency processing is derived using the following formula: (28) In formula (28), This is the alignment coefficient between the normal vector and the light source vector. It is dimensionless, derived from the dot product operation, and its value ranges from [value missing]. A value closer to 1 indicates a higher alignment. This is the surface normal vector of the point cloud, which has no unit and is derived from the calculation of the point cloud normal vector; it is a unit vector. This is a light source vector, without units, derived from the light source configuration of the rendering engine, and is a unit vector. It is a 2-norm operator, unitless, used to calculate the magnitude of a vector; The topology after lighting and shadow consistency processing is unitless and originates from the alignment of normals and light sources. This is a unitless operator for handling lighting consistency, used to adjust the lighting and shadow effects of the topology based on the alignment coefficient. The original topology is unitless and originates from the geometric topology of the point cloud. The control logic of formula (28) uses the point cloud density information as the precision constraint, performs a normalized dot product operation on the surface normal and the light source vector of the original topology, and calculates the alignment coefficient between the two. Then, through the light and shadow consistency processing operator, the lighting calculation parameters of the topology are dynamically adjusted according to the alignment coefficient to correct the light and shadow discontinuity and light and dark misalignment caused by topology reconstruction. Finally, a topology with global light and shadow uniformity and natural light and dark transition is generated, providing a high-fidelity light and shadow rendering foundation for the localization reconstruction of UE scenes.

[0183] Based on the point cloud density information, the light source vector and surface normal are aligned during the topology reconstruction process to handle the lighting consistency of the topology. In one implementation, the topology reconstruction algorithm first constructs a mesh based on density information. For example, when simulating indoor lighting, the surface normal vector is calculated for high-density point cloud regions. Then, the light source vector is aligned using the vector dot product, such as adjusting the normal direction to minimize the angle with the light source vector, thereby handling lighting consistency. This alignment process involves iterative optimization to ensure natural shadow casting, avoid uneven lighting, and improve the overall realism of the rendering.

[0184] Step S740: Map the topology structure with processed lighting and shadow consistency to the original rendering pipeline, and synthesize a stable high-fidelity scene model in the original rendering pipeline through multi-level progressive texture mapping.

[0185] A stable high-fidelity scene model is derived using the following formula: (29) In formula (29), To stabilize the high-fidelity scene model, it is unitless and derived from the results of multi-level progressive texture mapping, serving as the final UE scene migration and reconstruction model; It is a multi-level progressive texture mapping operator with no unit, used to fuse the processed topology with the texture to improve the model fidelity; The topology after the lighting and shadow consistency processing is unitless and comes from the calculation result of formula (28); This is the raw rendering pipeline, without units, derived from the dedicated rendering pipeline of step S500; The multi-level progressive texture is unitless and originates from the texture mapping configuration. It contains texture layers of different resolutions to improve rendering fidelity and efficiency. The control logic of formula (29) uses the topological structure after the lighting and shadow consistency processing as the geometric reference and maps it back to the original rendering pipeline. Then, through the multi-level progressive texture mapping operator, combined with the preset multi-level progressive texture configuration, the scene is gradually blended and restored from coarse to fine textures. Finally, a stable high-fidelity scene model is generated. This control logic realizes the seamless migration of UE scenes in domestic engines, while ensuring the geometric integrity, texture fineness and high uniformity of the reconstructed model's rendering performance.

[0186] The topology, after processing for lighting consistency, is mapped to the original rendering pipeline. A stable, high-fidelity scene model is then synthesized within this pipeline using multi-level progressive texture mapping. In one implementation, multi-level progressive texture mapping involves a step-by-step application from low to high resolution; for example, mapping a base texture first, then overlaying detail layers to synthesize a stable, high-fidelity scene model. This synthesis process is performed in real-time within the original rendering pipeline, ensuring scene stability. This method achieves efficient rendering.

[0187] Please see Figure 2 This invention provides a domestic migration and reconstruction system for UE scenes based on Gaussian sputtering, used to implement the aforementioned domestic migration and reconstruction method for UE scenes based on Gaussian sputtering. It includes an image sequence and camera parameter acquisition module 10, a preprocessed image sequence and initial point cloud acquisition module 20, an initial 3D model acquisition module 30, an optimized 3D model acquisition module 40, a preliminary rendering framework acquisition module 50, an interactive compatible rendering framework acquisition module 60, and a stable high-fidelity scene model acquisition module 70. The image sequence and camera parameter acquisition module 10 is used to plan an automatic roaming trajectory in a virtual reality environment using a virtual camera matrix, outputting original image sequences and video streams from multiple perspectives, simultaneously extracting camera intrinsic and extrinsic parameters, and using a coordinate system mapping algorithm to transform the left-handed coordinate system to the right-handed coordinate system to obtain aligned image sequences and camera parameters. The preprocessed image sequence and initial point cloud acquisition module 20 is used to perform denoising and sharpening processing on the aligned image sequences and camera parameters, simultaneously importing depth map normal maps to generate initial point clouds, accelerating convergence speed, and obtaining preprocessed image sequences and initial point clouds. The 3D model acquisition module 30 is used to input preprocessed image sequences and initial point clouds using a 3D reconstruction algorithm, express color and light and shadow using spherical harmonic functions, and control the ellipsoid shape using a covariance matrix to obtain an initial 3D model. The optimized 3D model acquisition module 40 is used to iteratively optimize the initial 3D model through an optimization renderer to minimize the error between the rendered image and the original virtual reality environment image. If the optimization error is lower than a preset threshold, the iteration stops, resulting in an optimized 3D model. The preliminary rendering framework acquisition module 50 is used to export the optimized 3D model into a point cloud standard format, build a dedicated rendering pipeline in the spatial rendering engine, and use block sorting technology to process the rendering order, resulting in a preliminary rendering framework. The interactive compatible rendering framework acquisition module 60 is used to integrate Alpha blending technology based on the preliminary rendering framework to achieve a real-time rendering frame rate, acquire auxiliary collision mesh data, and obtain an interactive compatible rendering framework. The stable high-fidelity scene model acquisition module 70 is used to adjust the block sorting parameters and integrate point cloud density information to process light and shadow consistency if rendering performance fluctuations are detected in the interactive compatible rendering framework, resulting in a stable high-fidelity scene model.

[0188] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering, characterized in that, Includes the following steps: S100: Plans an automatic roaming trajectory in a virtual reality environment using a virtual camera matrix, outputs original image sequences and video streams from multiple perspectives, extracts camera intrinsic and extrinsic parameters synchronously, and uses a coordinate system mapping algorithm to transform the left-handed coordinate system to the right-handed coordinate system to obtain the aligned image sequence and camera parameters. S200. For the aligned image sequence and camera parameters, perform noise reduction and sharpening processing, and simultaneously import the depth map normal map to generate an initial point cloud to accelerate the convergence speed and obtain the preprocessed image sequence and the initial point cloud. S300. Using a 3D reconstruction algorithm, the preprocessed image sequence and the initial point cloud are input, color and light are expressed by spherical harmonic functions, and the covariance matrix controls the ellipsoid shape to obtain the initial 3D model. S400. The initial 3D model is iteratively optimized by the optimizer to minimize the error between the rendered image and the original virtual reality environment image. If the optimization error is lower than a preset threshold, the iteration is stopped to obtain the optimized 3D model. S500. Export the optimized 3D model as a point cloud standard format, build a dedicated rendering pipeline in the spatial rendering engine, and use block sorting technology to process the rendering order to obtain a preliminary rendering framework. S600. Based on the preliminary rendering framework, Alpha blending technology is incorporated to achieve real-time rendering frame rate, obtain auxiliary collision mesh data, and obtain an interactive compatible rendering framework. S700. If rendering performance fluctuations are detected in the interactive compatible rendering framework, the block sorting parameters are adjusted, the point cloud density information is fused to process the light and shadow consistency, and a stable high-fidelity scene model is obtained.

2. The method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering according to claim 1, characterized in that, Step S100 includes: S110. Plan an automatic roaming trajectory in a virtual reality environment using a virtual camera matrix, and trigger multi-view rendering instructions based on the trajectory data stream to obtain the original image sequence; S120. Extract the projection matrix of the original image sequence, determine the intrinsic parameter matrix and extrinsic parameter matrix of the original image sequence, and obtain the parameter set; S130. Use a transformation operator to perform a spatial transformation on the extrinsic parameter matrix in the parameter set to obtain the aligned camera extrinsic parameter matrix; S140. Perform geometric correction on the original image sequence based on the aligned camera extrinsic parameter matrix to obtain the aligned image sequence and camera parameters.

3. The method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering according to claim 2, characterized in that, Step S200 includes: S210. The aligned image sequence is processed by a median filter operator to remove pixel noise, resulting in a denoised image sequence. S220. Extract grayscale gradient features from the denoised image sequence and use the Laplacian operator to calculate edge contrast to obtain a sharpened image sequence. S230. The sharpened image sequence is synchronously imported into the depth texture and normal vector, and back-projected through the camera matrix to determine the set of spatial coordinates. S240. Construct geometric topological relationships based on the set of spatial coordinates. If the projection residual is lower than a preset iteration threshold, generate an initial point cloud. S250. The initial point cloud and the sharpened image sequence are sampled and matched, and dense reconstruction is performed by verifying the mapping accuracy to obtain the preprocessed image sequence and the initial point cloud.

4. The method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering according to claim 1, characterized in that, Step S300 includes: S310. Determine the covariance matrix based on the geometric topological relationship between the preprocessed image sequence and the initial point cloud, and obtain the ellipsoidal morphological features corresponding to the covariance matrix; S320. The initial point cloud is spatially constrained using the ellipsoidal morphological features, and the local illumination intensity is fitted using the spherical harmonic function to determine the spherical harmonic components, thereby obtaining a color and light distribution model. S330. Establish a color mapping function based on the color and light distribution model, map the spherical harmonic components to the surface corresponding to the ellipsoidal morphological features, and obtain the initial three-dimensional model surface texture with light and shadow features. S340. The ellipsoidal shape is finely adjusted by performing surface texture adjustment on the initial three-dimensional model, and the eigenvalues ​​of the covariance matrix are matched with the preset geometric constraints to obtain the initial three-dimensional model.

5. The method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering according to claim 4, characterized in that, Step S400 includes: S410. Obtain the ellipsoidal morphological features and spherical harmonic components contained in the initial 3D model, and perform rasterization processing through the preset viewpoint transformation matrix to obtain the rendered image. S420. Based on the pixel-by-pixel comparison between the rendered image and the original virtual reality environment image, calculate the pixel residual and determine the mean square error of the current iteration cycle. S430. Based on the mean square error, the gradient operator for the ellipsoidal morphological features and the spherical harmonic components is calculated using the backpropagation algorithm to obtain the iteration step size for numerical updates. S440. Using the iteration step size, numerical corrections are performed on the spatial position of the initial three-dimensional model, the covariance matrix, and the color and light distribution model to obtain the corrected three-dimensional model. S450. If the optimization error generated by the corrected 3D model is lower than a preset threshold, the iteration process is stopped and the optimized 3D model is obtained.

6. The method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering according to claim 1, characterized in that, Step S500 includes: S510. Obtain the center coordinates and covariance matrix of the optimized 3D model, and transform the ellipsoidal morphological features of the optimized 3D model through the attribute mapping mechanism to obtain the point cloud standard format. S520. Perform spatial index construction according to the point cloud standard format, and load the point cloud standard format into the spatial rendering engine using a memory alignment mechanism; S530. In the spatial rendering engine, tile division is performed according to the viewpoint transformation matrix to determine the screen space area corresponding to the tile division. S540. A depth sorting algorithm is used to arrange the rendering order of the point cloud standard format for the screen space region to obtain an ordered vertex stream. S550 performs vertex shading and pixel blending on an ordered vertex stream through a dedicated rendering pipeline to obtain a preliminary rendering framework.

7. The method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering according to claim 6, characterized in that, Step S600 includes: S610. Extract pixel color values ​​and vertex attributes from the initial rendering framework, perform linear interpolation on the pixel color values ​​using a preset blending factor, and determine the blended pixel sequence in conjunction with depth cache data; S620. Perform frame rate threshold monitoring on the mixed pixel sequence. If the current rendering frequency is lower than the preset frequency, perform downsampling processing on the mixed pixel sequence through a dynamic thinning algorithm to obtain a high-frequency rendering stream. S630. Extract geometric boundary features based on vertex coordinates in the high-frequency rendering stream, generate a collision voxel set using a topology reconstruction algorithm, and obtain auxiliary collision mesh data. S640. Map the auxiliary collision mesh data to the interaction command response matrix, and match the interaction command response matrix with the rendering pipeline state through a compatible interface to obtain an interactive compatible rendering framework.

8. The method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering according to claim 1, characterized in that, Step S700 includes: S710. If rendering performance fluctuations are detected in the interactive compatible rendering framework, adjust the block sorting values ​​according to the geometric boundary features in the depth cache data. The adjusted block sorting parameters are obtained using the following formula: ； in, These are the adjusted block sorting parameters. Adjust the operator for block sorting parameters. These are the original block sorting parameters. Geometric boundary features This represents the value for rendering performance fluctuations. S720. Spatial clustering of vertex coordinates is performed using the block sorting values ​​to obtain point cloud density information; S730. Align the light source vector with the surface normal during the topology reconstruction process based on the point cloud density information to handle the lighting consistency of the topology. S740. The topology structure with the light and shadow consistency processed is mapped to the original rendering pipeline, and a stable high-fidelity scene model is synthesized in the original rendering pipeline through multi-level progressive texture mapping.

9. The method for domestic migration and reconstruction of UE scenes based on Gaussian sputtering according to claim 8, characterized in that, In step S720, the point cloud density information and spatial clustering results are obtained using the following formulas: ； in, For point cloud density information, The number of points in a single spatial cluster. The area of ​​a single spatial cluster. It is a spatial clustering operator with no unit. It is a standard point cloud format. These are the adjusted block sorting parameters, without units. This is the result of spatial clustering.

10. A UE scene localization migration and reconstruction system based on Gaussian sputtering, used to implement the UE scene localization migration and reconstruction method based on Gaussian sputtering as described in any one of claims 1 to 9, characterized in that, include: The image sequence and camera parameter acquisition module is used to plan an automatic roaming trajectory in a virtual reality environment through a virtual camera matrix, output the original image sequence and video stream from multiple perspectives, extract the camera intrinsic and extrinsic parameters synchronously, and use a coordinate system mapping algorithm to transform the left-handed coordinate system to the right-handed coordinate system to obtain the aligned image sequence and camera parameters. The preprocessed image sequence and initial point cloud acquisition module is used to perform denoising and sharpening processing on the aligned image sequence and camera parameters, and simultaneously import the depth map normal map to generate the initial point cloud, thereby accelerating the convergence speed and obtaining the preprocessed image sequence and the initial point cloud. The initial 3D model acquisition module is used to input the preprocessed image sequence and the initial point cloud using a 3D reconstruction algorithm, express color and light and shadow using spherical harmonic functions, and control the ellipsoid shape using a covariance matrix to obtain the initial 3D model. The optimized 3D model acquisition module is used to iteratively optimize the initial 3D model through the optimized renderer to minimize the error between the rendered image and the original virtual reality environment image. If the optimization error is lower than a preset threshold, the iteration stops and an optimized 3D model is obtained. The preliminary rendering framework acquisition module is used to export the optimized 3D model into a point cloud standard format, build a dedicated rendering pipeline in the spatial rendering engine, and use block sorting technology to process the rendering order to obtain the preliminary rendering framework. An interactive compatible rendering framework acquisition module is used to integrate Alpha blending technology into the preliminary rendering framework, achieve real-time rendering frame rate, acquire auxiliary collision mesh data, and obtain an interactive compatible rendering framework. The stable high-fidelity scene model acquisition module is used to adjust the block sorting parameters and process the light and shadow consistency by fusing point cloud density information if rendering performance fluctuations are detected in the interactive compatible rendering framework, so as to obtain a stable high-fidelity scene model.

Images