Virtual reality scene three-dimensional reconstruction method and system based on multi-source data fusion

Abstract

The invention discloses a virtual reality scene three-dimensional reconstruction method and system based on multi-source data fusion, and relates to the technical field of underground commercial and traffic integrated space scene three-dimensional reconstruction. The system constructs a multi-dimensional data set through synchronous acquisition of point cloud, images, poses, positioning and semantic text information; a three-dimensional convolutional neural network is adopted to establish an AI reconstruction fusion model, and a three-dimensional model is generated; the system calculates a structural integrity evaluation coefficient JGPG, a fusion consistency evaluation coefficient RHPG and a VR interaction adaptability evaluation coefficient VRJH, compares the coefficients with corresponding thresholds, triggers an optimization strategy, and finally completes model adaptability optimization and packaging output, thereby improving the reduction degree and interaction performance of a virtual reality scene.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction of underground commercial and transportation integrated space scenes, and specifically to a method and system for three-dimensional reconstruction of virtual reality scenes based on multi-source data fusion. Background Art

[0002] With the accelerating pace of urbanization, the development and utilization of urban underground space is becoming increasingly widespread, especially in core urban areas. The integrated construction of underground commercial facilities and transportation hubs has gradually become an important means to improve land use efficiency and alleviate surface transportation pressure. These integrated underground commercial and transportation spaces are complex in structure and diverse in function, involving multiple key aspects such as personnel flow organization, equipment layout, safe evacuation, signage and navigation, and placing higher demands on spatial planning, operation and maintenance management, and emergency response.

[0003] Currently, the management and visualization of urban underground spaces primarily rely on two-dimensional drawings, BIM models, or traditional three-dimensional modeling methods. While these methods have some feasibility for static display or during the construction phase, they face significant challenges: data uniformity, high structural complexity, difficulty in updating and maintenance, and a poor interactive experience. In recent years, the rapid development of technologies such as artificial intelligence, three-dimensional reconstruction, and virtual reality has provided new approaches for achieving high-precision, dynamically updated three-dimensional modeling and immersive display of underground spaces. However, existing technologies still lack a complete system for integrated underground commercial and transportation spaces that can integrate multi-source heterogeneous data, perform structural semantic modeling, and generate VR interactive output. In particular, effective solutions for dynamic spatial recognition, structural status assessment, and multi-role immersive interaction have yet to be developed. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides a method and system for three-dimensional reconstruction of virtual reality scenes based on multi-source data fusion to solve the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method and system for 3D reconstruction of virtual reality scenes based on multi-source data fusion, including a data acquisition module, a data processing module, an AI reconstruction fusion model establishment module, a structure reconstruction monitoring module, a fusion monitoring module, a VR interaction monitoring module and a real scene output module; The data acquisition module is used to complete the synchronous acquisition of multi-source heterogeneous data in the underground commercial and transportation integrated space scene through multiple devices, including: point cloud and image information, device posture and positioning data, semantic visual elements and text information; The data processing module is used to perform spatiotemporal alignment, denoising fusion and dense modeling on the collected data to establish a first data set; fuse IMU and UWB data to perform trajectory estimation and sensor registration to establish a second data set; and use target detection, semantic segmentation and OCR recognition technology to extract semantic elements and encode and annotate them to establish a third data set; The AI ​​reconstruction fusion model building module is used to train and extract features based on a three-dimensional convolutional neural network, combining spatial structure fusion data with semantic annotation data, to build an AI reconstruction fusion model for three-dimensional model generation and multimodal feature analysis, and to support subsequent evaluation and strategy output of structural integrity, fusion consistency, and VR interaction adaptability; The structural reconstruction monitoring module is used to monitor the geometric structural integrity and density of the three-dimensional reconstructed model, calculate and obtain the structural integrity evaluation coefficient JGPG, and compare and analyze it with the first threshold Q1 to determine whether the structural reconstruction is complete. If it is incomplete, a strategy is given; The fusion monitoring module is used to monitor the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information, calculate the fusion consistency evaluation coefficient RHPG, and compare and analyze it with the second threshold Q2 to determine whether the fusion error rate is qualified. If it is unqualified, a strategy is given; The VR interaction monitoring module is used to monitor the visualization degree, interaction performance and rendering smoothness of the three-dimensional model in the virtual reality environment in real time, calculate and obtain the VR interaction adaptability evaluation coefficient VRJH, and compare and analyze it with the third threshold Q3 to determine whether the three-dimensional model meets the VR interaction performance requirements. If not, a strategy is given; The real scene output module is used to optimize the adaptability of the three-dimensional reconstructed model and generate processing instructions based on the structural integrity, fusion consistency and VR interaction adaptability evaluation results and corresponding strategies, so as to achieve the final adjustment and output packaging of the model.

[0006] Preferably, the data acquisition module includes a spatial structure image acquisition unit, a posture positioning information acquisition unit and a scene semantic annotation information acquisition unit; The spatial structure image acquisition unit is used to collect point cloud data of the underground commercial and integrated space scene by installing a three-dimensional laser radar device, including the structural outlines of walls, columns, corridors and escalators; by installing an RGB-D camera device, it collects depth maps and synchronized color image data to form dense structure and texture alignment information; and by using a patrol robot equipped with a high-definition wide-angle camera, it collects spatial wide-angle environmental images and key visual feature area images; The position and positioning information acquisition unit is used to obtain the three-axis acceleration, angular velocity and attitude information of the acquisition device through the integrated inertial measurement device for SLAM synchronization and sensor alignment; by deploying UWB positioning equipment, the position coordinates in the underground enclosed space are collected; The scene semantic annotation information acquisition unit is used to collect semantic visual object images of shop doors, guide signs, and entrance and exit numbers by deploying image recognition cameras; and to collect text content, route markings, and notice statements on subway station signs by installing OCR recognition cameras.

[0007] Preferably, the data processing module includes a first data processing unit, a second data processing unit and a third data processing unit; The first data processing unit is used to use time synchronization and coordinate unification technology to align the timestamps and perform spatial coordinate transformation on the LiDAR point cloud, RGB-D depth map, and IMU pose data; use point cloud filtering and simplification algorithms to remove noise and redundancy from the structural point cloud data; use image depth fusion and voxel stitching algorithms to combine the depth map, color image, and point cloud to form a dense three-dimensional structure representation with consistent texture and geometry, and establish a first data set; The second data processing unit is used to estimate the device trajectory and posture information using the IMU point cloud combined with SLAM technology; fuse UWB positioning data to construct a continuous and verifiable trajectory line; use a multi-sensor calibration algorithm to perform time and space alignment between sensors and establish a second data set; The third data processing unit is used to use the YOLO-Nano target detection algorithm to extract semantic targets of shop doors, guide signs, and entrance and exit signs; use DeepLabv3 semantic segmentation technology to obtain scene semantic masks and label maps; use Tesseract OCR technology to identify Chinese or English route instructions, numbers, and notices in the image; spatially annotate and category encode all semantic information, generate a structured semantic map and interactive element index table, and establish a third data set.

[0008] Preferably, the AI ​​reconstruction fusion model establishment module is used to construct a three-dimensional convolutional neural network initial model using a three-dimensional convolutional neural network, and train and test the three-dimensional convolutional neural network initial model with data from the first data set, the second data set, and the third data set, and use the trained three-dimensional convolutional neural network initial model as the AI ​​reconstruction fusion model. At the same time, the intermediate layer output of the spatial structure fusion data and the scene semantic annotation data is used as a feature vector to identify structural features and semantic information, and the AI ​​reconstruction fusion model is trained and tested through the acquired feature information, and the trained AI reconstruction fusion model is used for three-dimensional model construction and multimodal feature analysis, support structure integrity assessment, fusion consistency assessment, and VR interaction adaptability calculation and strategy output.

[0009] Preferably, the structure reconstruction monitoring module includes a first calculation unit and a first analysis unit; The first calculation unit is used to monitor the geometric structural integrity and density of the three-dimensional reconstructed model. Combined with the data of the first data set, after dimensionless processing, the structural integrity evaluation coefficient JGPG is calculated and obtained. The formula is as follows: ; Where, represents the total area of ​​the closed structure surface, At represents the total surface area of ​​the model, represents the number of reconstructed voxels, Vt represents the theoretical voxel range, and w1 and w2 represent weight coefficients.

[0010] Preferably, the first analysis unit is used to preset a first threshold Q1 in advance, and compare and analyze the structural integrity assessment coefficient JGPG with the first threshold Q1, and obtaining the first assessment result includes: When the structural integrity assessment coefficient JGPG ≥ the first threshold Q1, it means that the structure is completely rebuilt and continues to be monitored; When the structural integrity assessment coefficient JGPG is less than the first threshold Q1, it indicates that the structural reconstruction is incomplete, triggering the first warning instruction and generating the first strategy: re-collecting the missing area data, calling the hole detection algorithm and deep inference technology for shape completion.

[0011] Preferably, the fusion monitoring module includes a second calculation unit and a second analysis unit; The second calculation unit is used to monitor the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information. After dimensionless processing, the fusion consistency evaluation coefficient RHPG is calculated by combining the data of the first data set, the second data set, and the third data set. The formula is as follows: ; Where Oe represents the semantic edge overlap rate, which is the ratio of the length of the overlap between the semantic boundary line of the image and the geometric edge line of the point cloud; Ef represents the mean fusion error, which is the spatial offset distance after the point cloud and the image are aligned; w3 and w4 represent weight coefficients; The second analysis unit is configured to preset a second threshold Q2 in advance, and compare and analyze the fusion consistency evaluation coefficient RHPG with the second threshold Q2 to obtain a second evaluation result, including: When the fusion consistency evaluation coefficient RHPG ≥ the second threshold Q2, it means that the fusion error rate is qualified, the registration is successful, and continuous monitoring is required; When the fusion consistency evaluation coefficient RHPG is less than the second threshold Q2, it means that the fusion error rate is unqualified and the registration fails, triggering the second warning instruction and generating the second strategy: automatically correct the image semantic annotation deviation, perform point cloud re-registration and trajectory reconstruction, and recalculate until the fusion consistency evaluation coefficient RHPG is greater than or equal to the second threshold Q2.

[0012] Preferably, the VR interaction monitoring module includes a third calculation unit and a third analysis unit; The third calculation unit is used to analyze the adaptability of the three-dimensional reconstructed model in the immersive virtual reality scene, and to monitor in real time the visualization degree, interactive performance, and rendering smoothness of the three-dimensional model in the virtual reality environment. The VR interaction adaptability evaluation coefficient VRJH is calculated after dimensionless processing by combining the data of the first data set and the third data set. The formula is as follows: ; ; Where Rr represents the visualization of the interactive viewpoint, Fs represents the frame rate stability coefficient, Rc represents the completeness rate of semantic interactive object recognition response, w5, w6 and w7 represent weight coefficients, represents the standard deviation, Indicates the average frame rate; The third analysis unit is used to preset a third threshold Q3 in advance, and compare and analyze the VR interaction adaptability evaluation coefficient VRJH with the third threshold Q3, and obtain a third evaluation result including: When the VR interaction adaptability evaluation coefficient VRJH ≥ the third threshold Q3, it means that the 3D model meets the VR interaction performance requirements, and there are no potential interaction problems in VR operation, and continuous monitoring is required; When the VR interaction adaptability evaluation coefficient VRJH is less than the third threshold Q3, it means that the three-dimensional model does not meet the VR interaction performance requirements, and there are potential interaction problems in VR operation, which triggers the third warning instruction and generates the third strategy: automatically analyze the blocked areas in the user's perspective path, adjust the POV parameters, and implement transparency processing for the invisible areas; perform local texture on the interactive hotspot areas, increase the resolution by 20%, and automatically switch the Level of Detail based on the model complexity and device performance, improving the frame rate stability by 10% and the loading efficiency by 10%; for semantic interaction objects with delayed response or insufficient recognition in the current model, including buttons, house numbers and prompt signs, increase the confidence re-recognition and optimize the interactive areas by 20%.

[0013] Preferably, the real scene output module is used to optimize the adaptability and generate results of the three-dimensional reconstructed model based on the judgment results of the structural integrity evaluation coefficient JGPG, the fusion consistency evaluation coefficient RHPG and the VR interaction adaptability evaluation coefficient VRJH, combined with the corresponding generation strategies, to complete the final adjustment and output packaging of the model.

[0014] Preferably, the method for 3D reconstruction of a virtual reality scene based on multi-source data fusion comprises the following steps: Step 1: In an underground commercial and transportation integrated space, multiple devices are used to synchronously collect multi-source heterogeneous data, including: point cloud and image information, device posture and positioning data, semantic visual elements, and text information; Step 2: Perform spatiotemporal alignment, denoising fusion, and dense modeling on the collected data to establish the first dataset; fuse IMU and UWB data for trajectory estimation and sensor registration to establish the second dataset; use target detection, semantic segmentation, and OCR recognition technology to extract semantic elements and encode and annotate them to establish the third dataset; Step 3: Based on a 3D convolutional neural network, combined with spatial structure fusion data and semantically annotated data for training and feature extraction, an AI reconstruction fusion model is constructed for 3D model generation and multimodal feature analysis. This model also supports the subsequent evaluation and strategy output of structural integrity, fusion consistency, and VR interaction adaptability. Step 4: By monitoring the geometric structure integrity and density of the 3D reconstructed model, the structural integrity evaluation coefficient JGPG is calculated and compared with the first threshold Q1 to determine whether the structural reconstruction is complete. If not, a strategy is given; Step 5: By monitoring the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information, the fusion consistency evaluation coefficient RHPG is calculated and compared with the second threshold Q2 to determine whether the fusion error rate is qualified. If it is unqualified, a strategy is given; Step 6: By real-time monitoring the visualization, interactive performance, and rendering smoothness of the 3D model in the virtual reality environment, the VR interaction adaptability evaluation coefficient VRJH is calculated and compared with the third threshold Q3 to determine whether the 3D model meets the VR interaction performance requirements. If not, a strategy is given; Step 7: Based on the structural integrity, fusion consistency, and VR interaction adaptability assessment results and corresponding strategies, the 3D reconstructed model is optimized for adaptability and processing instructions are generated to achieve final adjustment and output packaging of the model.

[0015] The present invention provides a method and system for 3D reconstruction of virtual reality scenes based on multi-source data fusion. It has the following beneficial effects: (1) This virtual reality scene 3D reconstruction method and system based on multi-source data fusion realizes high-precision 3D reconstruction of underground commercial and transportation integrated space by fusing point cloud, image, IMU, UWB and semantic visual information. It can dynamically reflect structural changes and environmental updates, improve the timeliness and accuracy of spatial modeling, and overcome the technical bottleneck of traditional 2D drawings and static BIM models that are difficult to update and have poor expression.

[0016] (2) This method and system for 3D reconstruction of virtual reality scenes based on multi-source data fusion has built a unified data fusion interface and format conversion module, which can perform standardized preprocessing of heterogeneous data such as point clouds, images, inertial navigation, positioning and semantic labels, and is compatible with a variety of 3D engines and VR platforms to achieve highly compatible output for different terminal devices, thereby improving the system's versatility and deployment flexibility in different application scenarios.

[0017] (3) This method and system for 3D reconstruction of virtual reality scenes based on multi-source data fusion adopts 3D convolutional neural networks and multimodal feature fusion technology, which can deeply fuse structural geometry and semantic annotation information into modeling. Through the fusion consistency evaluation mechanism RHPG, it ensures a high degree of matching between semantic boundaries and structural features in the 3D model, providing a clear semantic expression basis for subsequent applications such as navigation guidance and safety management.

[0018] (4) This virtual reality scene 3D reconstruction method and system based on multi-source data fusion uses the VR interaction adaptability evaluation mechanism VRJH to dynamically analyze the model's visibility, frame rate stability, and semantic response effect in VR, automatically adjust the interactive perspective and detail rendering strategy, effectively improve the immersive visualization experience, and meet the operational fluency and interaction efficiency of different user roles in VR scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flowchart of the block diagram of the virtual reality scene 3D reconstruction system based on multi-source data fusion of the present invention; Figure 2 Schematic diagram of the steps of the method for three-dimensional reconstruction of a virtual reality scene based on multi-source data fusion of the present invention. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] Example 1 See also Figure 1, the present invention provides a virtual reality scene three-dimensional reconstruction system based on multi-source data fusion, including a data acquisition module, a data processing module, an AI reconstruction fusion model establishment module, a structure reconstruction monitoring module, a fusion monitoring module, a VR interaction monitoring module and a real scene output module; The data acquisition module is used to complete the synchronous acquisition of multi-source heterogeneous data in the underground commercial and transportation integrated space scene through multiple devices, including: point cloud and image information, device posture and positioning data, semantic visual elements and text information; The data processing module is used to perform spatiotemporal alignment, denoising fusion and dense modeling on the collected data to establish a first data set; fuse IMU and UWB data to perform trajectory estimation and sensor registration to establish a second data set; and use target detection, semantic segmentation and OCR recognition technology to extract semantic elements and encode and annotate them to establish a third data set; The AI ​​reconstruction fusion model building module is used to train and extract features based on a three-dimensional convolutional neural network, combining spatial structure fusion data with semantic annotation data, to build an AI reconstruction fusion model for three-dimensional model generation and multimodal feature analysis, and to support subsequent evaluation and strategy output of structural integrity, fusion consistency, and VR interaction adaptability; The structural reconstruction monitoring module is used to monitor the geometric structural integrity and density of the three-dimensional reconstructed model, calculate and obtain the structural integrity evaluation coefficient JGPG, and compare and analyze it with the first threshold Q1 to determine whether the structural reconstruction is complete. If it is incomplete, a strategy is given; The fusion monitoring module is used to monitor the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information, calculate the fusion consistency evaluation coefficient RHPG, and compare and analyze it with the second threshold Q2 to determine whether the fusion error rate is qualified. If it is unqualified, a strategy is given; The VR interaction monitoring module is used to monitor the visualization degree, interaction performance and rendering smoothness of the three-dimensional model in the virtual reality environment in real time, calculate and obtain the VR interaction adaptability evaluation coefficient VRJH, and compare and analyze it with the third threshold Q3 to determine whether the three-dimensional model meets the VR interaction performance requirements. If not, a strategy is given; The real scene output module is used to optimize the adaptability of the three-dimensional reconstructed model and generate processing instructions based on the structural integrity, fusion consistency and VR interaction adaptability evaluation results and corresponding strategies, so as to achieve the final adjustment and output packaging of the model.

[0022] In this embodiment, through modular multi-source data acquisition and processing, AI reconstruction fusion and monitoring mechanisms, high-precision, real-time three-dimensional reconstruction and VR adaptation of the underground commercial and transportation integrated space can be achieved. Specifically, the data acquisition module can synchronously acquire multiple data sources, and perform precise spatiotemporal alignment and fusion processing on the data through the data processing module, ensuring the high precision and high consistency of the reconstructed model. At the same time, the structural reconstruction monitoring, fusion monitoring and VR interactive monitoring modules can evaluate the geometric integrity, data fusion consistency and interactive adaptability of the model in real time, ensuring the visualization and interactive performance of the three-dimensional reconstructed model in the virtual reality environment, and ultimately achieving a virtual scene output of the underground commercial and transportation integrated space with high precision, high adaptability and high interactivity.

[0023] Example 2 This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically, the data acquisition module includes a spatial structure image acquisition unit, a ,pose positioning information acquisition unit, and a scene semantic annotation ,information acquisition unit; The spatial structure image acquisition unit is used to collect point cloud data of the underground commercial and integrated space scene by installing a three-dimensional laser radar device, including the structural outlines of walls, columns, corridors and escalators; by installing an RGB-D camera device, it collects depth maps and synchronized color image data to form dense structure and texture alignment information; and by using a patrol robot equipped with a high-definition wide-angle camera, it collects spatial wide-angle environmental images and key visual feature area images; The position and positioning information acquisition unit is used to obtain the three-axis acceleration, angular velocity and attitude information of the acquisition device through the integrated inertial measurement device for SLAM synchronization and sensor alignment; by deploying UWB positioning equipment, the position coordinates in the underground enclosed space are collected; The scene semantic annotation information acquisition unit is used to collect semantic visual object images of shop doors, guide signs, and entrance and exit numbers by deploying image recognition cameras; and to collect text content, route markings, and notice statements on subway station signs by installing OCR recognition cameras.

[0024] In this embodiment, by integrating a variety of advanced data acquisition equipment, all-round and multi-dimensional data acquisition of the underground integrated commercial and transportation space is achieved. The spatial structure image acquisition unit uses three-dimensional lidar, RGB-D camera and inspection robot to collect point clouds, depth maps and environmental images with high precision, ensuring the dense structure and texture alignment of the scene, and providing rich geometric information for subsequent three-dimensional reconstruction. The posture positioning information acquisition unit uses inertial measurement unit and UWB positioning device to accurately obtain device position and posture information, effectively supporting SLAM synchronization and sensor alignment, and ensuring high accuracy and consistency of data. The scene semantic annotation information acquisition unit uses image recognition and OCR technology to accurately extract semantic information such as shop doorheads, signboards, entrance and exit numbers, enriching the semantic level of the scene and further enhancing the semantic understanding and sense of reality of the three-dimensional reconstruction model. These high-precision, multi-level data acquisition and integration have laid a solid foundation for subsequent three-dimensional reconstruction and virtual reality adaptation.

[0025] Example 3 This embodiment is explained in Example 2, please refer to Figure 1 ,Specifically, the data processing module includes a first data processing unit, a second data processing unit and a third data processing unit; The first data processing unit is used to use time synchronization and coordinate unification technology to align the timestamps and perform spatial coordinate transformation on the LiDAR point cloud, RGB-D depth map, and IMU pose data; use point cloud filtering and simplification algorithms to remove noise and redundancy from the structural point cloud data; use image depth fusion and voxel stitching algorithms to combine the depth map, color image, and point cloud to form a dense three-dimensional structure representation with consistent texture and geometry, and establish a first data set; The second data processing unit is used to estimate the device trajectory and posture information using the IMU point cloud combined with SLAM technology; fuse UWB positioning data to construct a continuous and verifiable trajectory line; use a multi-sensor calibration algorithm to perform time and space alignment between sensors and establish a second data set; The third data processing unit is used to use the YOLO-Nano target detection algorithm to extract semantic targets of shop doors, guide signs, and entrance and exit signs; use DeepLabv3 semantic segmentation technology to obtain scene semantic masks and label maps; use Tesseract OCR technology to identify Chinese or English route instructions, numbers, and notices in the image; spatially annotate and category encode all semantic information, generate a structured semantic map and interactive element index table, and establish a third data set.

[0026] In this embodiment, through multi-level data processing units, efficient integration and optimization of underground commercial and transportation integrated spatial data are achieved. The first data processing unit accurately connects LiDAR point cloud, RGB-D depth map and IMU pose data through time synchronization and coordinate unification technology, removes noise and redundancy, and forms a dense and texture- and geometry-consistent three-dimensional structure representation through image depth fusion and voxel stitching technology, providing high-quality basic data for subsequent reconstruction. The second data processing unit accurately estimates the device trajectory and pose information through the fusion of IMU point cloud, SLAM technology and UWB positioning data, ensuring the continuity and verifiability of the entire data acquisition process, and providing reliable support for the accurate modeling of three-dimensional space. The third data processing unit accurately extracts semantic targets, labels and text information in the scene through YOLO-Nano target detection, DeepLabv3 semantic segmentation and TesseractOCR text recognition technology, and performs spatial annotation and category encoding on them, generating a structured semantic map and interactive element index table, providing complete and efficient semantic data support for subsequent three-dimensional model and virtual reality adaptation.

[0027] Example 4 This embodiment is explained in Example 3, please refer to Figure 1 Specifically, the AI ​​reconstruction fusion model establishment module is used to construct a three-dimensional convolutional neural network initial model using a three-dimensional convolutional neural network, and train and test the three-dimensional convolutional neural network initial model with data from the first data set, the second data set, and the third data set, and use the trained three-dimensional convolutional neural network initial model as the AI ​​reconstruction fusion model. At the same time, the intermediate layer output of the spatial structure fusion data and the scene semantic annotation data is used as a feature vector to identify structural features and semantic information, and the AI ​​reconstruction fusion model is trained and tested through the acquired feature information, and the trained AI reconstruction fusion model is used for three-dimensional model construction and multimodal feature analysis, supporting structure integrity assessment, fusion consistency assessment, and VR interaction adaptability calculation and strategy output.

[0028] In this embodiment, an AI reconstruction fusion model is constructed using a three-dimensional convolutional neural network (3D CNN), thereby achieving efficient fusion of three-dimensional structure and semantic features. By combining spatial structure fusion data with scene semantic annotation data, the model can extract rich structural features and semantic information from multimodal information, and further optimize the training process through the feature vector output by the intermediate layer, thereby improving the accuracy and detail of three-dimensional reconstruction. The trained AI reconstruction fusion model can not only accurately construct a three-dimensional model, but also perform multimodal feature analysis, providing strong support for the evaluation of structural integrity, fusion consistency and VR interactive adaptability, and output corresponding strategies based on the evaluation results, thereby effectively improving the adaptability and interactive performance of the model, and ensuring high-quality reconstruction and virtual reality adaptation of the underground commercial and integrated space.

[0029] Example 5 This embodiment is explained in Example 4. Please refer to Figure 1 ,Specifically, the structure reconstruction monitoring module includes a first ,calculation unit and a first analysis unit; The first calculation unit is used to monitor the geometric structural integrity and density of the three-dimensional reconstructed model. Combined with the data of the first data set, after dimensionless processing, the structural integrity evaluation coefficient JGPG is calculated and obtained. The formula is as follows: ; Where, represents the total area of ​​the closed structure surface, At represents the total surface area of ​​the model, Represents the number of reconstructed voxels, Vt represents the theoretical voxel range, w1 and w2 represent weight coefficients, , ,and .

[0030] In this embodiment, the first calculation unit in the structural reconstruction monitoring module monitors the geometric integrity and density of the 3D reconstructed model in real time. This method accurately calculates the structural integrity evaluation coefficient JGPG, enhancing model quality control. This calculation method combines the data from the first dataset with dimensionless processing to analyze the model's surface area and voxel information to assess whether the reconstructed model meets geometric integrity requirements. By adjusting the weight coefficients w1 and w2, the evaluation accuracy can be optimized according to specific needs, enabling early identification of potential defects in the structural reconstruction process and providing an accurate basis for the generation of subsequent optimization strategies.

[0031] Example 6 This embodiment is explained in Example 5, please refer to Figure 1Specifically, the first analysis unit is used to preset a first threshold Q1 in advance, and compare and analyze the structural integrity assessment coefficient JGPG with the first threshold Q1, and obtain the first assessment result including: When the structural integrity assessment coefficient JGPG ≥ the first threshold Q1, it means that the structure is completely rebuilt and continues to be monitored; When the structural integrity assessment coefficient JGPG is less than the first threshold Q1, it indicates that the structural reconstruction is incomplete, triggering the first warning instruction and generating the first strategy: re-collecting the missing area data, calling the hole detection algorithm and deep inference technology for shape completion.

[0032] In this embodiment, the structural integrity assessment mechanism set up by the first analysis unit can automatically assess the structural integrity of the 3D reconstructed model. When the structural integrity assessment coefficient JGPG falls below the preset first threshold Q1, the system can immediately trigger an early warning mechanism and automatically generate an optimization strategy, such as re-collecting data in missing areas and combining a hole detection algorithm with deep inference technology to complete the shape, thereby effectively resolving structural defects in the model reconstruction. This mechanism improves the accuracy and automation level of the reconstruction process, ensures the integrity of the 3D reconstructed model, and quickly corrects problems when they are discovered, greatly improving the robustness of the system and reconstruction efficiency, as shown in the following table:

[0033] Example 7 This embodiment is explained in Example 4. Please refer to Figure 1 ,Specifically, the fusion monitoring module includes a second calculation unit and a second analysis unit; The second calculation unit is used to monitor the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information. After dimensionless processing, the fusion consistency evaluation coefficient RHPG is calculated by combining the data of the first data set, the second data set, and the third data set. The formula is as follows: ; Where Oe represents the semantic edge overlap rate, which is the ratio of the length of the overlap between the semantic boundary line of the image and the geometric edge line of the point cloud. Ef represents the mean fusion error, which is the spatial offset distance after the point cloud and the image are aligned. w3 and w4 represent weight coefficients. , and ; The second analysis unit is configured to preset a second threshold Q2 in advance, and compare and analyze the fusion consistency evaluation coefficient RHPG with the second threshold Q2 to obtain a second evaluation result, including: When the fusion consistency evaluation coefficient RHPG ≥ the second threshold Q2, it means that the fusion error rate is qualified, the registration is successful, and continuous monitoring is required; When the fusion consistency evaluation coefficient RHPG is less than the second threshold Q2, it means that the fusion error rate is unqualified and the registration fails, triggering the second warning instruction and generating the second strategy: automatically correct the image semantic annotation deviation, perform point cloud re-registration and trajectory reconstruction, and recalculate until the fusion consistency evaluation coefficient RHPG is greater than or equal to the second threshold Q2.

[0034] In this embodiment, the fusion consistency evaluation mechanism of the second calculation unit and the second analysis unit can accurately monitor the spatial registration effect and semantic matching degree of the point cloud and semantic image information. When the evaluation coefficient RHPG is lower than the preset second threshold Q2, the system automatically triggers an early warning and generates a strategy to automatically correct the image semantic annotation deviation, perform point cloud re-registration and trajectory reconstruction, and ensure the accurate fusion of image and point cloud data. This automated correction function improves the accuracy and consistency of model fusion, reduces the need for manual intervention, greatly enhances the stability and adaptability of the system, effectively avoids data deviations caused by registration failures, and ensures high-quality output of three-dimensional reconstruction results, as shown in the following table:

[0035] Example 8 This embodiment is explained in Example 7, please refer to Figure 1 ,Specifically, the VR interaction monitoring module includes a third ,calculation unit and a third analysis unit; The third calculation unit is used to analyze the adaptability of the three-dimensional reconstructed model in the immersive virtual reality scene, and to monitor in real time the visualization degree, interactive performance, and rendering smoothness of the three-dimensional model in the virtual reality environment. The VR interaction adaptability evaluation coefficient VRJH is calculated after dimensionless processing by combining the data of the first data set and the third data set. The formula is as follows: ; ; Where Rr represents the visualization of the interactive viewpoint, Fs represents the frame rate stability, Rc represents the completeness of the semantic interactive object recognition response, w5, w6 and w7 represent the weight coefficients, , , and , represents the standard deviation, Indicates the average frame rate; The third analysis unit is used to preset a third threshold Q3 in advance, and compare and analyze the VR interaction adaptability evaluation coefficient VRJH with the third threshold Q3, and obtain a third evaluation result including: When the VR interaction adaptability evaluation coefficient VRJH ≥ the third threshold Q3, it means that the 3D model meets the VR interaction performance requirements, and there are no potential interaction problems in VR operation, and continuous monitoring is required; When the VR interaction adaptability evaluation coefficient VRJH is less than the third threshold Q3, it means that the three-dimensional model does not meet the VR interaction performance requirements, and there are potential interaction problems in VR operation, which triggers the third warning instruction and generates the third strategy: automatically analyze the blocked areas in the user's perspective path, adjust the POV parameters, and implement transparency processing for the invisible areas; perform local texture on the interactive hotspot areas, increase the resolution by 20%, and automatically switch the Level of Detail based on the model complexity and device performance, improving the frame rate stability by 10% and the loading efficiency by 10%; for semantic interaction objects with delayed response or insufficient recognition in the current model, including buttons, house numbers and prompt signs, increase the confidence re-recognition and optimize the interactive areas by 20%.

[0036] In this embodiment, through the VR interaction adaptability evaluation mechanism of the third calculation unit and the third analysis unit, the adaptability of the three-dimensional reconstructed model in the virtual reality environment can be monitored and evaluated in real time, especially its visualization, interactive performance and rendering smoothness. When the evaluation coefficient VRJH is lower than the preset third threshold Q3, the system will automatically analyze the occlusion area in the user's viewing path, adjust the viewpoint parameters and implement transparency processing, while optimizing the resolution and frame rate stability of the interactive hotspot area. This automated optimization strategy not only improves the interactive experience in the virtual reality environment, but also ensures the adaptability and performance of the system on different devices, significantly improves the interactive smoothness and user experience of the model in VR, and reduces potential problems caused by hardware limitations or insufficient interaction, as shown in the following table:

[0037] Example 9 This embodiment is explained in Example 8, please refer to Figure 1 Specifically, the real scene output module is used to optimize the adaptability and generate results of the three-dimensional reconstructed model based on the judgment results of the structural integrity evaluation coefficient JGPG, the fusion consistency evaluation coefficient RHPG and the VR interaction adaptability evaluation coefficient VRJH, combined with the corresponding generation strategies, to complete the final adjustment and output packaging of the model.

[0038] In this embodiment, the adaptability optimization mechanism of the real-world scene output module enables intelligent adjustment and optimization of the 3D reconstructed model based on the structural integrity, fusion consistency, and VR interaction adaptability assessment results and their corresponding strategies. This module ensures that the model meets the predetermined requirements under the evaluation criteria of different dimensions, automatically adjusts defects or inconsistencies in the model, optimizes the structure and interaction performance, and achieves optimal adaptation in the virtual reality environment. This process not only improves the overall quality and stability of the 3D model, but also significantly enhances the user experience, ensuring the model's usability and interactive fluency in real-world applications.

[0039] Example 10 For a 3D reconstruction method of virtual reality scenes based on multi-source data fusion, please refer to Figure 2 , including the following steps: Step 1: In an underground commercial and transportation integrated space, multiple devices are used to synchronously collect multi-source heterogeneous data, including: point cloud and image information, device posture and positioning data, semantic visual elements, and text information; Step 2: Perform spatiotemporal alignment, denoising fusion, and dense modeling on the collected data to establish the first dataset; fuse IMU and UWB data for trajectory estimation and sensor registration to establish the second dataset; use target detection, semantic segmentation, and OCR recognition technology to extract semantic elements and encode and annotate them to establish the third dataset; Step 3: Based on a 3D convolutional neural network, combined with spatial structure fusion data and semantically annotated data for training and feature extraction, an AI reconstruction fusion model is constructed for 3D model generation and multimodal feature analysis. This model also supports the subsequent evaluation and strategy output of structural integrity, fusion consistency, and VR interaction adaptability. Step 4: By monitoring the geometric structure integrity and density of the 3D reconstructed model, the structural integrity evaluation coefficient JGPG is calculated and compared with the first threshold Q1 to determine whether the structural reconstruction is complete. If not, a strategy is given; Step 5: By monitoring the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information, the fusion consistency evaluation coefficient RHPG is calculated and compared with the second threshold Q2 to determine whether the fusion error rate is qualified. If it is unqualified, a strategy is given; Step 6: By real-time monitoring the visualization, interactive performance, and rendering smoothness of the 3D model in the virtual reality environment, the VR interaction adaptability evaluation coefficient VRJH is calculated and compared with the third threshold Q3 to determine whether the 3D model meets the VR interaction performance requirements. If not, a strategy is given; Step 7: Based on the structural integrity, fusion consistency, and VR interaction adaptability assessment results and corresponding strategies, the 3D reconstructed model is optimized for adaptability and processing instructions are generated to achieve final adjustment and output packaging of the model.

[0040] In this embodiment, through a precise seven-step process, the comprehensive application of multi-source data acquisition, AI reconstruction fusion model training, and a multi-dimensional evaluation mechanism enables the efficient construction and optimization of three-dimensional reconstruction models in underground integrated commercial and transportation spaces. By dynamically monitoring and adjusting the model's structural integrity, fusion consistency, and VR interactive adaptability, the present invention can automatically detect and correct potential problems at each stage of model construction, ensuring that the final output three-dimensional model achieves optimal results in terms of accuracy, interactivity, and user experience. This process greatly improves the model's stability and adaptability, especially in virtual reality applications, optimizes the model's interactive fluency and adaptability, and greatly enhances its application value in real-world scenarios.

[0041] The threshold is set to facilitate comparison. The size of the threshold depends on the amount of sample data and the number of bases set by technicians in this field for each set of sample data; as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0042] The above formulas are obtained by collecting a large amount of data and performing software simulation, and a formula close to the actual value is selected. The coefficients in the formula are set by those skilled in the art according to actual conditions. The above is only a preferred specific implementation method of the present invention, but the protection scope of the present invention is not limited to this. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, can make equivalent replacements or changes based on the technical solution and inventive concept of the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A 3D reconstruction system for virtual reality scenes based on multi-source data fusion, characterized by: It includes data acquisition module, data processing module, AI reconstruction fusion model building module, structure reconstruction monitoring module, fusion monitoring module, VR interactive monitoring module and real scene output module; The data acquisition module is used to complete the synchronous acquisition of multi-source heterogeneous data in the underground commercial and transportation integrated space scene through multiple devices, including: point cloud and image information, device posture and positioning data, semantic visual elements and text information; The data processing module is used to perform spatiotemporal alignment, denoising fusion and dense modeling on the collected data to establish a first data set; fuse IMU and UWB data to perform trajectory estimation and sensor registration to establish a second data set; and use target detection, semantic segmentation and OCR recognition technology to extract semantic elements and encode and annotate them to establish a third data set; The AI ​​reconstruction fusion model building module is used to train and extract features based on a three-dimensional convolutional neural network, combining spatial structure fusion data with semantic annotation data, to build an AI reconstruction fusion model for three-dimensional model generation and multimodal feature analysis, and to support subsequent evaluation and strategy output of structural integrity, fusion consistency, and VR interaction adaptability; The structural reconstruction monitoring module is used to monitor the geometric structural integrity and density of the three-dimensional reconstructed model, calculate and obtain the structural integrity evaluation coefficient JGPG, and compare and analyze it with the first threshold Q1 to determine whether the structural reconstruction is complete. If it is incomplete, a strategy is given; The fusion monitoring module is used to monitor the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information, calculate the fusion consistency evaluation coefficient RHPG, and compare and analyze it with the second threshold Q2 to determine whether the fusion error rate is qualified. If it is unqualified, a strategy is given; The VR interaction monitoring module is used to monitor the visualization degree, interaction performance and rendering smoothness of the three-dimensional model in the virtual reality environment in real time, calculate and obtain the VR interaction adaptability evaluation coefficient VRJH, and compare and analyze it with the third threshold Q3 to determine whether the three-dimensional model meets the VR interaction performance requirements. If not, a strategy is given; The real scene output module is used to optimize the adaptability of the three-dimensional reconstructed model and generate processing instructions based on the structural integrity, fusion consistency and VR interaction adaptability evaluation results and corresponding strategies, so as to achieve the final adjustment and output packaging of the model.

2. The virtual reality scene 3D reconstruction system based on multi-source data fusion according to claim 1, characterized in that: The data acquisition module includes a spatial structure image acquisition unit, a posture positioning information acquisition unit and a scene semantic annotation information acquisition unit; The spatial structure image acquisition unit is used to collect point cloud data of the underground commercial and integrated space scene by installing a three-dimensional laser radar device, including the structural outlines of walls, columns, corridors and escalators; by installing an RGB-D camera device, it collects depth maps and synchronized color image data to form dense structure and texture alignment information; and by using a patrol robot equipped with a high-definition wide-angle camera, it collects spatial wide-angle environmental images and key visual feature area images; The position and positioning information acquisition unit is used to obtain the three-axis acceleration, angular velocity and attitude information of the acquisition device through the integrated inertial measurement device for SLAM synchronization and sensor alignment; by deploying UWB positioning equipment, the position coordinates in the underground enclosed space are collected; The scene semantic annotation information acquisition unit is used to collect semantic visual object images of shop doors, guide signs, and entrance and exit numbers by deploying image recognition cameras; and to collect text content, route markings, and notice statements on subway station signs by installing OCR recognition cameras.

3. The virtual reality scene 3D reconstruction system based on multi-source data fusion according to claim 2, characterized in that: The data processing module includes a first data processing unit, a second data processing unit and a third data processing unit; The first data processing unit is used to use time synchronization and coordinate unification technology to perform timestamp alignment and spatial coordinate transformation on the LiDAR point cloud, RGB-D depth map and IMU pose data; Point cloud filtering and simplification algorithms are used to remove noise and redundancy from structural point cloud data. Image depth fusion and voxel stitching algorithms are used to combine depth maps, color images, and point clouds to form a dense 3D structure representation with consistent texture and geometry, and to establish the first dataset. The second data processing unit is used to estimate the device trajectory and posture information using IMU-point cloud combined SLAM technology; Fusion of UWB positioning data to construct continuous, verifiable trajectory lines; use of multi-sensor calibration algorithms to perform temporal and spatial registration between sensors and establish a second data set; The third data processing unit is used to use the YOLO-Nano target detection algorithm to extract semantic targets of shop doors, guide signs, and entrance and exit signs; use DeepLabv3 semantic segmentation technology to obtain scene semantic masks and label maps; use Tesseract OCR technology to identify Chinese or English route instructions, numbers, and notices in the image; spatially annotate and category encode all semantic information, generate a structured semantic map and interactive element index table, and establish a third data set.

4. The virtual reality scene 3D reconstruction system based on multi-source data fusion according to claim 3, characterized in that: The AI ​​reconstruction fusion model establishment module is used to use a three-dimensional convolutional neural network to construct an initial model of a three-dimensional convolutional neural network, and train and test the initial model of the three-dimensional convolutional neural network using data from the first data set, the second data set, and the third data set. The trained initial model of the three-dimensional convolutional neural network is used as the AI ​​reconstruction fusion model, and the intermediate layer output of the spatial structure fusion data and the scene semantic annotation data is used as a feature vector to identify structural features and semantic information. The AI ​​reconstruction fusion model is trained and tested through the acquired feature information, and the trained AI reconstruction fusion model is used for three-dimensional model construction and multimodal feature analysis, supporting structure integrity assessment, fusion consistency assessment, and VR interaction adaptability calculation and strategy output.

5. The virtual reality scene 3D reconstruction system based on multi-source data fusion according to claim 4, characterized in that: The structural reconstruction monitoring module includes a first calculation unit and a first analysis unit; The first calculation unit is used to monitor the geometric structural integrity and density of the three-dimensional reconstructed model. Combined with the data of the first data set, after dimensionless processing, the structural integrity evaluation coefficient JGPG is calculated and obtained. The formula is as follows: ； Where, represents the total area of ​​the closed structure surface, At represents the total surface area of ​​the model, represents the number of reconstructed voxels, Vt represents the theoretical voxel range, and w1 and w2 represent weight coefficients.

6. The virtual reality scene 3D reconstruction system based on multi-source data fusion according to claim 5, characterized in that: The first analysis unit is used to preset a first threshold Q1 in advance, and compare and analyze the structural integrity assessment coefficient JGPG with the first threshold Q1, and obtain a first assessment result including: When the structural integrity assessment coefficient JGPG ≥ the first threshold Q1, it means that the structure is completely rebuilt and continues to be monitored; When the structural integrity assessment coefficient JGPG is less than the first threshold Q1, it indicates that the structural reconstruction is incomplete, triggering the first warning instruction and generating the first strategy: re-collecting the missing area data, calling the hole detection algorithm and deep inference technology for shape completion.

7. The virtual reality scene 3D reconstruction system based on multi-source data fusion according to claim 4, characterized in that: The fusion monitoring module includes a second calculation unit and a second analysis unit; The second calculation unit is used to monitor the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information. After dimensionless processing, the fusion consistency evaluation coefficient RHPG is calculated by combining the data of the first data set, the second data set, and the third data set. The formula is as follows: ； Where Oe represents the semantic edge overlap rate, which is the ratio of the length of the overlap between the semantic boundary line of the image and the geometric edge line of the point cloud; Ef represents the mean fusion error, which is the spatial offset distance after the point cloud and the image are aligned; w3 and w4 represent weight coefficients; The second analysis unit is configured to preset a second threshold Q2 in advance, and compare and analyze the fusion consistency evaluation coefficient RHPG with the second threshold Q2 to obtain a second evaluation result, including: When the fusion consistency evaluation coefficient RHPG ≥ the second threshold Q2, it means that the fusion error rate is qualified, the registration is successful, and continuous monitoring is required; When the fusion consistency evaluation coefficient RHPG is less than the second threshold Q2, it means that the fusion error rate is unqualified and the registration fails, triggering the second warning instruction and generating the second strategy: automatically correct the image semantic annotation deviation, perform point cloud re-registration and trajectory reconstruction, and recalculate until the fusion consistency evaluation coefficient RHPG is greater than or equal to the second threshold Q2.

8. The virtual reality scene 3D reconstruction system based on multi-source data fusion according to claim 7, characterized in that: The VR interaction monitoring module includes a third calculation unit and a third analysis unit; The third calculation unit is used to analyze the adaptability of the three-dimensional reconstructed model in the immersive virtual reality scene, and to monitor in real time the visualization degree, interactive performance, and rendering smoothness of the three-dimensional model in the virtual reality environment. The VR interaction adaptability evaluation coefficient VRJH is calculated after dimensionless processing by combining the data of the first data set and the third data set. The formula is as follows: ； ； Where Rr represents the visualization of the interactive viewpoint, Fs represents the frame rate stability coefficient, Rc represents the completeness rate of semantic interactive object recognition response, w5, w6 and w7 represent weight coefficients, represents the standard deviation, Indicates the average frame rate; The third analysis unit is used to preset a third threshold Q3 in advance, and compare and analyze the VR interaction adaptability evaluation coefficient VRJH with the third threshold Q3, and obtain a third evaluation result including: When the VR interaction adaptability evaluation coefficient VRJH ≥ the third threshold Q3, it means that the 3D model meets the VR interaction performance requirements, and there are no potential interaction problems in VR operation, and continuous monitoring is required; When the VR interaction adaptability evaluation coefficient VRJH is less than the third threshold Q3, it means that the three-dimensional model does not meet the VR interaction performance requirements, and there are potential interaction problems in VR operation, which triggers the third warning instruction and generates the third strategy: automatically analyze the blocked areas in the user's perspective path, adjust the POV parameters, and implement transparency processing for the invisible areas; perform local texture on the interactive hotspot areas, increase the resolution by 20%, and automatically switch the Level of Detail based on the model complexity and device performance, improving the frame rate stability by 10% and the loading efficiency by 10%; for semantic interaction objects with delayed response or insufficient recognition in the current model, including buttons, house numbers and prompt signs, increase the confidence re-recognition and optimize the interactive areas by 20%.

9. The virtual reality scene 3D reconstruction system based on multi-source data fusion according to claim 8, characterized in that: The real scene output module is used to optimize the adaptability and generate results of the three-dimensional reconstructed model based on the judgment results of the structural integrity evaluation coefficient JGPG, the fusion consistency evaluation coefficient RHPG and the VR interaction adaptability evaluation coefficient VRJH, combined with the corresponding generation strategies, to complete the final adjustment and output packaging of the model.

10. A method for 3D reconstruction of a virtual reality scene based on multi-source data fusion, comprising the system for 3D reconstruction of a virtual reality scene based on multi-source data fusion according to any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: In an underground commercial and transportation integrated space, multiple devices are used to synchronously collect multi-source heterogeneous data, including: point cloud and image information, device posture and positioning data, semantic visual elements, and text information; Step 2: Perform spatiotemporal alignment, denoising fusion, and dense modeling on the collected data to establish the first dataset; fuse IMU and UWB data for trajectory estimation and sensor registration to establish the second dataset; use target detection, semantic segmentation, and OCR recognition technology to extract semantic elements and encode and annotate them to establish the third dataset; Step 3: Based on a 3D convolutional neural network, combined with spatial structure fusion data and semantically annotated data for training and feature extraction, an AI reconstruction fusion model is constructed for 3D model generation and multimodal feature analysis. This model also supports the subsequent evaluation and strategy output of structural integrity, fusion consistency, and VR interaction adaptability. Step 4: By monitoring the geometric structure integrity and density of the 3D reconstructed model, the structural integrity evaluation coefficient JGPG is calculated and compared with the first threshold Q1 to determine whether the structural reconstruction is complete. If not, a strategy is given; Step 5: By monitoring the spatial registration effect and semantic matching degree between the point cloud structure and the semantic image information, the fusion consistency evaluation coefficient RHPG is calculated and compared with the second threshold Q2 to determine whether the fusion error rate is qualified. If it is unqualified, a strategy is given; Step 6: By real-time monitoring the visualization, interactive performance, and rendering smoothness of the 3D model in the virtual reality environment, the VR interaction adaptability evaluation coefficient VRJH is calculated and compared with the third threshold Q3 to determine whether the 3D model meets the VR interaction performance requirements. If not, a strategy is given; Step 7: Based on the structural integrity, fusion consistency, and VR interaction adaptability assessment results and corresponding strategies, the 3D reconstructed model is optimized for adaptability and processing instructions are generated to achieve final adjustment and output packaging of the model.

Images