Historic building protection display method and system based on virtual reality

Abstract

The invention discloses an ancient building protection display method and system based on virtual reality, and the method comprises the steps: obtaining user position coordinates and orientation angle data collected by a high-precision position tracking sensor in real time, mapping the data to a corresponding position of a virtual scene, dynamically updating a visual angle in the virtual scene according to the movement and steering of a user, and displaying the visual angle in the virtual scene. The immersive experience of being personally on the scene is realized; a physical rendering technology is adopted to carry out vivid simulation on illumination, materials, textures and the like of the virtual scene, and through technologies of pre-calculation of an illumination map, environment shielding and the like, the real-time rendering calculation amount is reduced, and the rendering efficiency and the visual quality of the virtual scene are improved; various displacement interaction modes including traditional rocker movement, gesture control, body inclination and the like are designed, operation habits of different users are met, user interaction behaviors are analyzed and modeled through a machine learning algorithm, interaction parameters and response modes are adjusted in a self-adaptive mode, and personalized displacement interaction experience is provided.

Description

Technical Field

[0001] The invention belongs to the field of information technology, and in particular relates to a method and system for protecting and displaying ancient buildings based on virtual reality. Background Art

[0002] In the virtual reality-based ancient building protection and display system, displacement technology is the key to enabling users to freely explore in virtual scenes. However, there are some technical difficulties and contradictions in the displacement of virtual scenes. First, in order to obtain an immersive experience, the system needs to capture the user's position and direction changes in real time and accurately, and map them to the virtual scene. This places high demands on the sensitivity and response speed of the sensor. Secondly, displacement boundaries and collision detection in virtual scenes are also a major challenge. In the real world, people can judge the position and distance of objects through senses such as vision and touch to avoid collisions. However, in virtual scenes, due to the lack of real tactile feedback, users may misjudge the distance, resulting in crossing objects or crossing boundaries. In addition, the virtual scenes in the ancient building display system usually contain a large number of fine architectural details and decorative elements. How to optimize the rendering efficiency of the scene while ensuring realism and reduce the jamming and delay during the displacement process is also an urgent problem to be solved. Finally, the height, body shape and behavioral habits of different users vary greatly. How to design a set of adaptive, flexible and changeable displacement interaction methods to meet the needs of different users is also a topic worthy of in-depth discussion. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for protecting and displaying ancient buildings based on virtual reality.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A method for displaying ancient buildings based on virtual reality, comprising:

[0006] Step S1, mapping the collected user position coordinates and orientation angle data to the corresponding position of the virtual scene, and dynamically updating the viewing angle in the virtual scene according to the user's movement and turning;

[0007] Step S2, preprocessing the virtual scene to obtain a spatial division result of dividing the virtual scene into multiple small areas;

[0008] Step S3: predict and compensate for the user's displacement and rotational motion, fuse and optimize the sensor data, and obtain position and direction estimation results with less noise interference;

[0009] Step S4: Perform detailed geometric description and spatial division of buildings and obstacles in the virtual scene, perform ray detection in real time during the user's displacement, and restrict or guide the user's movement based on the detection results.

[0010] Preferably, in step S2, the virtual scene is preprocessed using an octree space division algorithm to obtain a space division result that divides the virtual scene into multiple small areas, and a bounding box is established for each small area. When the user performs displacement interaction, the small area where the user's current position is located is determined, and collision detection and rendering are performed only on virtual objects within the small area.

[0011] Preferably, in step S3, an inertial navigation algorithm is used to predict and compensate for the user's displacement and rotational motion using data collected by an accelerometer and a gyroscope, and a Kalman filter algorithm is used to fuse and optimize the sensor data to obtain position and direction estimation results with less noise interference.

[0012] Preferably, in step S4, based on the high-precision three-dimensional collision detection model, the buildings and obstacles in the virtual scene are finely geometrically described and spatially divided, and during the user's displacement, ray detection is performed in real time to determine the distance and collision relationship between the user and the virtual objects in the virtual scene, and the user's movement is restricted or guided based on the detection results.

[0013] The present invention also provides an ancient building protection and display system based on virtual reality, comprising:

[0014] A first processing device, used to map the collected user position coordinates and orientation angle data to corresponding positions in the virtual scene, and dynamically update the viewing angle in the virtual scene according to the user's movement and turning;

[0015] A second processing device is used to pre-process the virtual scene to obtain a spatial division result of dividing the virtual scene into a plurality of small areas;

[0016] The third processing device is used to predict and compensate for the displacement and rotational motion of the user, fuse and optimize the sensor data, and obtain the position and direction estimation results with less noise interference;

[0017] The fourth processing device is used to make a detailed geometric description and spatial division of buildings and obstacles in the virtual scene, perform ray detection in real time during the user's displacement, and restrict or guide the user's movement based on the detection results.

[0018] Preferably, the second processing device uses an octree space division algorithm to pre-process the virtual scene, obtains a space division result that divides the virtual scene into multiple small areas, and establishes a bounding box for each small area. When the user performs displacement interaction, the small area where the user's current position is located is determined, and collision detection and rendering are only performed on virtual objects within the small area.

[0019] Preferably, the third processing device predicts and compensates for the user's displacement and rotational movement by using the data collected by the accelerometer and gyroscope through an inertial navigation algorithm, and adopts a Kalman filter algorithm to fuse and optimize the sensor data to obtain position and direction estimation results with less noise interference.

[0020] Preferably, the fourth processing device performs detailed geometric description and spatial division of buildings and obstacles in the virtual scene based on a high-precision three-dimensional collision detection model, performs real-time ray detection during the user's displacement, determines the distance and collision relationship between the user and virtual objects in the virtual scene, and restricts or guides the user's movement based on the detection results.

[0021] The present invention uses a high-precision position tracking sensor to collect user position and orientation data in real time, and maps it to the virtual scene to achieve dynamic perspective update. In order to improve rendering efficiency, the present invention uses an octree space partitioning algorithm to pre-process the virtual scene, and combines inertial navigation and Kalman filtering algorithms to optimize position estimation. At the same time, a high-precision three-dimensional collision detection model is constructed, and ray detection is performed in real time to determine the distance and collision relationship between the user and the virtual object. In terms of rendering, the present invention uses physics-based technology, combined with pre-calculated light mapping and other methods to improve visual quality. In addition, the present invention also designs a variety of displacement interaction modes, and adaptively adjusts the interaction parameters through a machine learning algorithm to provide users with a personalized experience. The comprehensive application of these technologies significantly improves the immersion and interaction effects of virtual scenes, bringing users a more realistic and smooth virtual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0023] Figure 1 The present invention is a flowchart of the virtual reality ancient building protection and display method. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Embodiment 1:

[0027] like Figure 1 As shown, an embodiment of the present invention provides a method for protecting and displaying ancient buildings based on virtual reality, comprising:

[0028] S101. Obtain user position coordinates and orientation angle data collected in real time by a high-precision position tracking sensor, map the data to a corresponding position in a virtual scene, and dynamically update the viewing angle in the virtual scene according to the user's movement and turning to achieve an immersive experience.

[0029] Acquire the user's position coordinates and orientation angle data collected in real time by the high-precision position tracking sensor, and map the acquired position coordinate data to the corresponding three-dimensional space position in the virtual scene through coordinate conversion. Determine the viewing angle in the virtual scene based on the user's orientation angle data, and synchronize the viewing angle in the virtual scene with the user's actual orientation. Track the changes in the user's position coordinates in real time, dynamically adjust the viewing angle position in the virtual scene according to the change amplitude and direction of the position coordinates, and achieve synchronous update of the virtual scene and user movement. Acquire the angle change data of the user's turning action, dynamically adjust the viewing angle of the virtual scene according to the angle change amplitude and direction, and achieve real-time synchronization of the viewing angle in the virtual scene with the user's turning. Synchronize the viewing angle position and orientation in the virtual scene with the user's actual position and orientation in real time, and generate a three-dimensional virtual scene image consistent with the user's perspective through the rendering engine to provide an immersive visual experience. Render the environmental sound effects and sounds in the virtual scene through the audio engine, and dynamically adjust the sound source position and volume according to the changes in the user's position and orientation to create realistic sound effects and enhance immersion. The integrated visual rendering and audio rendering of virtual scenes provide users with an immersive experience, enabling users to obtain a sensory experience in the virtual scene that is highly consistent with the real scene, enhancing the user's sense of involvement and immersion.

[0030] Furthermore, the high-precision position tracking sensor uses an inertial measurement unit (IMU) and a visual odometer (Visual Odometry) to collect the user's position coordinates (x, y, z) and orientation angle (yaw, pitch, roll) data in real time at a frequency of 100Hz. The acquired position coordinate data is multiplied by the virtual scene's scaling factor of 5, and then added to the virtual scene's origin offset (1000, 1000, 0), and mapped to the corresponding three-dimensional space position in the virtual scene. According to the user's orientation angle data, the view orientation in the virtual scene is determined to be consistent with it through rotation matrix transformation. The system tracks the user's position coordinate changes in real time. When the position coordinate change exceeds the threshold of 10cm or the change direction exceeds the angle of the last change direction by more than 30°, the virtual scene's view position is dynamically adjusted and updated synchronously at a refresh rate of 60 frames per second. When the angle change of the user's turning action is detected to exceed 5°, the virtual scene's view orientation is dynamically adjusted at a ratio of 1° per degree according to the direction of the angle change, so as to achieve real-time synchronization of the view orientation and the user's turning. By synchronizing the perspective position and orientation in the virtual scene with the user's actual position and orientation in real time, the physically based rendering (PBR) engine is used to generate high-quality three-dimensional virtual scene images that are consistent with the user's perspective at a rendering frame rate of 90 frames per second, providing an immersive visual experience. At the same time, the audio engine uses the head-related transfer function (HRTF) algorithm to perform real-time reverberation and positioning of environmental sound effects and sounds in the virtual scene according to changes in the user's position and orientation, dynamically adjust the sound source position and volume attenuation, create a realistic three-dimensional stereo effect, and enhance the spatial and directional sense of the sound. Ultimately, through the seamless combination of visual rendering and audio rendering, users are provided with a full-range, multi-sensory immersive experience, enabling them to obtain an audio-visual experience in the virtual scene that is highly consistent with the real scene, greatly enhancing the user's sense of substitution and immersion, and bringing an immersive and realistic experience.

[0031] S102, using an octree space division algorithm to pre-process the virtual scene, obtain a space division result of dividing the virtual scene into multiple small areas, and establish a bounding box for each small area; when the user performs displacement interaction, determine the small area where the user's current position is located, and only perform collision detection and rendering on virtual objects within the small area.

[0032] The octree algorithm is used to pre-process the virtual scene for spatial division, and the virtual scene is divided into multiple small areas. A bounding box is established for each small area to obtain the spatial division result and bounding box information. The pre-established virtual scene model data is obtained, and the virtual scene model data is divided into multiple sub-model data according to the spatial division result, and each sub-model data corresponds to a small area. For each small area, the virtual object in the area is determined according to the bounding box information, and a mapping relationship between the virtual object and the bounding box is established. When the user performs displacement interaction, the user's current position coordinates are obtained in real time, the small area where the coordinates are located is determined, and the user's current area is determined. The virtual object data in the user's current area is obtained, and the virtual object range that needs to be collided is quickly locked according to the mapping relationship between the virtual object and the bounding box. The hierarchical bounding box algorithm is used to perform collision detection on the virtual object determined in step 5, determine whether the user collides with the virtual object, and obtain the collision detection result. According to the collision detection result, the frustum culling algorithm and the occlusion culling algorithm are used to render the virtual object in the user's current area, and the screen display in the virtual scene is updated in real time.

[0033] Furthermore, the octree algorithm is used to pre-process the virtual scene for spatial division, and the virtual scene is divided into multiple small areas, each of which is 10m×10m×10m in size. A bounding box is established for each small area, and the minimum and maximum vertex coordinates of the bounding box are calculated to obtain the spatial division result and bounding box information. The pre-established virtual scene model data is obtained, and the virtual scene model data is divided into multiple sub-model data according to the spatial division result. Each sub-model data corresponds to a small area, and the sub-model data includes the geometric information, texture information, and position information of the virtual object in the area. For each small area, according to the bounding box information, the AABB collision detection algorithm is used to determine the virtual object in the area, and a mapping relationship between the virtual object and the bounding box is established. The ID of the virtual object is associated with the ID of the bounding box for quick search during collision detection. When the user performs displacement interaction, the coordinates of the user's current position are obtained in real time. By comparing the coordinates with the bounding box, the small area where the coordinates are located is determined to determine the current area of ​​the user. Get the virtual object data in the area where the user is currently located, and quickly lock the range of virtual objects that need to be collided according to the mapping relationship between virtual objects and bounding boxes, reducing the calculation amount of collision detection to 1 / 8 of the original. Use the hierarchical bounding box algorithm to establish a hierarchical bounding box tree for virtual objects that need to be collided. Through the hierarchical structure of the tree, perform collision detection layer by layer from top to bottom to determine whether the user collides with the virtual object and obtain the collision detection result. This algorithm can reduce the time complexity of collision detection to O(logn). According to the collision detection results, use the frustum culling algorithm and the occlusion culling algorithm to judge the visibility of virtual objects in the area where the user is currently located, and eliminate objects that are not within the frustum and are occluded, reduce the number of objects rendered, improve rendering efficiency, and update the screen display in the virtual scene in real time to ensure that the user's visual experience is smooth and not stuck during movement.

[0034] According to the collision detection results, the frustum culling algorithm and the occlusion culling algorithm are used to render the virtual objects in the user's current area, and update the screen display in the virtual scene in real time.

[0035] The spatial position information and viewing direction information of the area where the user is currently located are obtained to determine the range of virtual objects that need to be rendered. For the virtual objects that need to be rendered, a collision detection algorithm is used to determine whether there is a collision between the virtual objects. If there is a collision between the virtual objects, the position and posture of the virtual objects are adjusted according to the collision detection result to avoid overlapping or penetration between the virtual objects. After completing the collision detection and virtual object position adjustment, a frustum culling algorithm is used to determine whether the virtual object is within the user's field of view. If the virtual object is outside the frustum, the virtual object is removed from the rendering queue. For virtual objects within the frustum, an occlusion culling algorithm is further used to determine whether there is an occlusion relationship between the virtual objects. By comparing the distance between the virtual object and the viewpoint, the occluded virtual object is removed from the rendering queue. According to the results of frustum culling and occlusion culling, the virtual object set that needs to be rendered is determined, and the virtual object set is passed to the graphics rendering engine for rendering calculation and screen generation. The virtual scene screen generated by the graphics rendering engine is merged with the real environment image to obtain an augmented reality display screen.

[0036] Further, after obtaining the spatial position information and viewing direction information of the user's current area, the system can use the frustum culling algorithm to filter out virtual objects within the user's field of view according to the preset field of view parameters (such as a field of view angle of 90 degrees and a viewing distance of 20 meters) as a set of candidate objects that need to be rendered. Next, for the candidate object set, the system uses a collision detection algorithm based on a hierarchical bounding box to quickly determine whether there is a collision between objects by comparing the positional relationship between the bounding boxes of virtual objects. If it is detected that the bounding boxes of object A and object B intersect, the precise grid models of the two objects are further compared to calculate the minimum distance between the models. When the minimum distance is less than the preset collision threshold (such as 0.5 meters), it is determined that the two objects collide. For virtual objects that collide, the system calculates the collision response based on the physical engine, and adjusts the position and posture of the object by applying separation force and friction force, so that it is correctly separated in the next frame to avoid overlapping or penetration. After completing the collision detection and object position adjustment, the system uses an occlusion culling algorithm based on a depth buffer to determine the occlusion relationship between objects by comparing the distance between the virtual object and the viewpoint. Specifically, the system sorts objects from front to back according to their distance from the viewpoint, and removes objects that are far away and obscured from the rendering queue, thereby reducing unnecessary rendering calculations. Finally, the removed virtual object set is passed to the graphics rendering engine, which uses GPU acceleration technology to perform rendering operations such as vertex transformation, lighting calculation, and texture mapping to generate a realistic virtual scene. The system then fuses the virtual scene with the real environment image obtained through the camera in real time, and uses algorithms such as visual inertial odometer to achieve accurate alignment of virtual objects with real scenes, ultimately presenting an immersive augmented reality experience.

[0037] S103. By using an inertial navigation algorithm, the data collected by sensors such as accelerometers and gyroscopes are comprehensively utilized to predict and compensate for the user's displacement and rotational motion; the sensor data are fused and optimized using a Kalman filter algorithm to obtain position and direction estimation results with less noise interference.

[0038] According to the inertial navigation algorithm, the raw data collected by sensors such as accelerometers and gyroscopes are obtained. For the obtained raw data, the Kalman filter algorithm is used to eliminate noise interference and obtain optimized sensor data. According to the optimized sensor data, the user's displacement and rotational motion information is obtained through integral operation. If the user's displacement or rotational motion information exceeds the preset threshold, it is judged that the user has made corresponding movements, and the direction and amplitude of the movement are obtained. According to the direction and amplitude of the user's movement, the inertial navigation algorithm is used to predict the position and direction to obtain the user's current position coordinates and orientation. The user's position coordinates and orientation are used as input, and data fusion is performed through the Kalman filter algorithm to obtain the position and direction estimation results with less noise interference. According to the position and direction estimation results, the user's displacement and rotational motion are compensated to obtain accurate user motion trajectory and direction information.

[0039] Furthermore, in the inertial navigation system, the raw data is first collected through sensors such as accelerometers and gyroscopes. For example, the accelerometer collects three-axis acceleration data every 0.1 seconds, and the gyroscope collects three-axis angular velocity data every 0.1 seconds. Then, the Kalman filter algorithm is used to eliminate noise interference from the collected raw data. By establishing the state equation and observation equation, using a priori estimation and a posteriori estimation, the state variables are continuously updated iteratively to obtain the optimized sensor data. For example, after filtering, the noise level of the acceleration data is reduced from ±1m / s to ±2m / s. 2 Reduced to ±05m / s 2 . Next, based on the optimized sensor data, the user's displacement and rotation information is obtained through integration operations. For example, by integrating the acceleration data, the user's displacement in three directions is obtained, and by integrating the angular velocity data, the user's rotation angle is obtained. If the user's displacement exceeds 5m or the rotation angle exceeds 30°, it is determined that the user has made corresponding movements, and the direction and amplitude of the movement are obtained. According to the direction and amplitude of the user's movement, the inertial navigation algorithm is used to predict the position and direction. For example, by superimposing the current position coordinates with the displacement, the user's new position coordinates are obtained, and by superimposing the current orientation with the rotation angle, the user's new orientation is obtained. The predicted user position coordinates and orientation are used as observation values, and are fused with the prior estimation through the Kalman filter algorithm to obtain the position and direction estimation results with less noise interference. Finally, based on the position and direction estimation results, the user's displacement and rotational movement are compensated. For example, when rendering a virtual scene, the position and direction of the virtual camera are adjusted in real time according to the estimated position and orientation, so as to obtain accurate user motion trajectory and direction information, thereby improving the accuracy and fluency of the virtual reality experience.

[0040] S104, constructing a high-precision three-dimensional collision detection model, and performing detailed geometric description and spatial division of buildings, obstacles, etc. in the virtual scene; during the user's displacement process, performing ray detection in real time to determine the distance and collision relationship between the user and virtual objects in the virtual scene, and restricting or guiding the user's movement according to the detection results.

[0041] According to the pre-established high-precision three-dimensional virtual scene model, the fine geometric description data and spatial division data of the buildings and obstacles in the model are obtained; when the user is detected to have moved, the user's current position information is obtained, and the three-dimensional coordinates of the user's position are mapped in the three-dimensional virtual scene model; starting from the user's position coordinates, the ray detection algorithm is used to detect the virtual objects in the user's line of sight in real time in the three-dimensional virtual scene model; the result data of the ray detection is obtained to determine whether the distance between the user and the detected virtual object is less than the preset collision distance threshold; if so, it is determined that the user collides with the virtual object, and the user's movement direction and speed are restricted according to the collision relationship data to prevent the user from passing through the virtual object; if not, the spatial position data of the virtual object is obtained, and the user's movement direction is guided according to the preset guidance rules so that it faces the target virtual object; according to the results of the user's movement restriction and guidance, the user's position coordinates in the three-dimensional virtual scene model are updated, and the updated position information is fed back to the user to provide an immersive interactive experience.

[0042] Furthermore, first, a 3D modeling software such as 3ds Max or Maya is used to perform high-precision modeling of the target scene, obtain fine geometric description data of buildings and obstacles in the scene, and use the octree algorithm to spatially divide the scene to generate spatial index data of the scene. When the system detects that the user's position has changed, the user's current 3D coordinates are calculated by the inertial navigation algorithm and mapped to the virtual scene model. Starting from the user's position coordinates, the system uses a ray casting algorithm to detect virtual objects in the user's line of sight in real time at a frequency of 60 frames per second, and calculates the distance between the object and the user. If the distance is less than the collision threshold of 5 meters, it is determined that a collision has occurred. According to the material properties of the virtual object (such as hardness, elasticity, etc.), a physical engine such as PhysX is used to calculate the user's speed and direction after the collision, and it is restricted to avoid penetration. If no collision occurs, the user's movement direction is guided by the potential field method according to the preset guidance rules (such as always keeping the target object in the center of the field of view, and the distance from the target object is within 2 meters, etc.), so that it is directed toward the target object. Finally, based on the results of user movement restrictions and guidance, the Kalman filter algorithm is used to update the user's position, and the updated position information is fed back to the user at a frequency of 30 frames per second, providing an immersive interactive experience through multi-channel feedback such as vision, hearing, and touch.

[0043] S105. Use physically based rendering technology to realistically simulate the lighting, materials, textures, etc. of the virtual scene; reduce the amount of real-time rendering calculations and improve the rendering efficiency and visual quality of the virtual scene by pre-calculating light maps, ambient occlusion, and other technologies.

[0044] According to the geometric model and material properties of the virtual scene, physically based rendering algorithms such as path tracing and photon mapping are used to simulate the propagation and reflection of light in the scene to obtain realistic lighting effects. For each material in the scene, according to its physical properties such as roughness, metalness and other parameters, microsurface models such as the Cook-Torrance model are used to simulate the reflection and refraction of light by the material to obtain the real material appearance. For the textures in the scene, anisotropic filtering, mipmap and other technologies are used to sample the textures at different viewing angles and distances to avoid aliasing and noise, and improve the clarity and realism of the textures. For static objects in the scene, the light maps under different lighting conditions are pre-calculated, and the maps are directly sampled during real-time rendering to avoid repeated calculations and improve rendering efficiency. In the pre-calculation stage of the scene, the ambient occlusion technology is used to calculate the occlusion of each shading point on the surrounding geometry, and it is used as an additional occlusion factor during real-time rendering to enhance the spatial sense and realism of the scene. According to the complexity and real-time requirements of the virtual scene, the level of detail technology is used to render models and textures of different precisions at different viewing angles and distances, improving rendering efficiency while ensuring visual quality. The rendering process is optimized, and multi-threading, GPU parallel computing and other technologies are used to make full use of hardware resources, reduce rendering time, and ensure the real-time interactive performance of the virtual scene.

[0045] Furthermore, in virtual scene rendering, a path tracing algorithm is used to simulate light propagation. For different materials, such as metal materials, the Cook-Torrance microsurface model is used, and the roughness parameter is set to 2 and the metalness parameter is set to 8 to simulate the reflection and refraction effects of light on the metal surface. For texture filtering of anisotropic materials such as wood, an anisotropic filtering algorithm is used, and an 8-level texture pyramid is generated using mipmap technology. The appropriate texture resolution is dynamically selected according to the viewing angle and distance to ensure efficient texture sampling and avoid aliasing. For static objects in the scene, such as buildings, their light maps under 5 different lighting conditions are pre-calculated. Spherical harmonic function compressed light maps are used for real-time rendering, which reduces memory usage by 80%. In the pre-calculation stage, ambient occlusion technology is used to calculate the occlusion factor of each shading point by ray sampling in 64 directions. It is used as an additional occlusion factor in real-time rendering and mixed into the shading result to enhance the spatial sense of the scene. According to the complexity of the scene, the level detail technology is used to divide the scene into 8 detail levels, and the corresponding level models and textures are rendered under different viewing angles. Without losing visual quality, the rendering efficiency is increased by 5 times. In the optimization of the rendering process, multi-threaded parallel computing technology is used to realize the parallelization of the rendering pipeline on an 8-core CPU, shortening the rendering time by 70%, ensuring the real-time interactive performance of the virtual scene.

[0046] In the pre-calculation stage of the scene, ambient occlusion technology is used to calculate the occlusion of each shading point by the surrounding geometry, and it is used as an additional occlusion factor during real-time rendering to enhance the scene's sense of space and reality.

[0047] According to the pre-established scene model, the position information of each shading point in the scene and the position information of the surrounding geometric bodies are obtained. For each shading point, the occlusion relationship between it and the surrounding geometric bodies is calculated to obtain the occlusion factor of each shading point. The calculated occlusion factor of each shading point is stored in a preset data structure and associated with the corresponding shading point. When rendering the scene in real time, the position information of each shading point in the current rendering frame is obtained. For each shading point, its corresponding occlusion factor is obtained from the preset data structure. According to the obtained occlusion factor, the shading calculation of each shading point is adjusted so that its light intensity is multiplied by the occlusion factor to simulate the ambient occlusion effect. The adjusted shading result is applied to the current rendering frame, the rendering effect of the scene is updated, and the spatial sense and realism of the scene are enhanced.

[0048] Furthermore, in the scene pre-calculation stage, using technologies such as ray casting, a large number of rays are projected around each shading point as the starting point, and the occlusion relationship is calculated by intersection with the scene geometry. For example, 1024 rays can be projected at each shading point, distributed on a hemisphere centered on the point, to obtain the intersection information of the rays and the surrounding geometry. The closer the intersection is to the shading point, the greater the occlusion factor in the corresponding direction. The calculated occlusion factor is stored in the texture map, each pixel corresponds to a shading point, and the pixel value is the occlusion factor of the point. Considering the spatial correlation of occlusion information, the occlusion map can be mipmap processed, and occlusion information of different precisions can be stored at different mipmap levels. During real-time rendering, the occlusion factor is sampled from the pre-calculated occlusion map by trilinear interpolation based on the UV coordinates of the current shading point and its depth value in the camera space. For example, for a shading point with a UV coordinate of (3,8) and a depth value of 5, interpolation is performed in the adjacent 4 pixels of the 3*width row and 8*height column of the occlusion map, and then the depth value is combined to interpolate in the occlusion map of two adjacent mipmap levels to finally obtain the occlusion factor of the point. Multiply the sampled occlusion factor by the light intensity of the point to obtain the final shading result, which can simulate the real ambient occlusion effect. For example, for a shading point with an occlusion factor of 8, its final light intensity is 8 times the original light intensity. When rendering, the point will show a dark effect blocked by surrounding objects, which enhances the spatial sense of the scene. By applying shading calculations based on pre-calculated occlusion information to all shading points in the scene, a virtual scene with real ambient occlusion is finally rendered.

[0049] S106. Design a variety of displacement interaction modes, including traditional joystick movement, gesture control, body tilt, etc., to meet the operating habits of different users; analyze and model user interaction behaviors through machine learning algorithms, adaptively adjust interaction parameters and response methods, and provide personalized displacement interaction experience.

[0050] The user's displacement interaction operation data is obtained, including operation parameters in various interaction modes such as joystick movement, gesture control, and body tilt. The obtained user displacement interaction operation data is preprocessed to remove noise and abnormal data and extract key feature parameters. The user's displacement interaction operation data is analyzed using a clustering algorithm, and users are divided into different groups according to their operation habits and preferences. For each user group, a displacement interaction response model is established using a decision tree algorithm to determine the optimal interaction parameters and response methods under different interaction modes. When the user performs a displacement interaction operation, its operation data is obtained in real time, and the interaction parameters and response methods are dynamically adjusted according to the interaction response model of the user group to which it belongs. The user's displacement interaction operation data is continuously collected, and the user group division and interaction response model are regularly updated to adapt to changes in user operation habits. Provide each user with a personalized displacement interaction experience, and recommend the most suitable interaction mode and parameter settings according to their operation habits and preferences to improve interaction efficiency and user satisfaction.

[0051] Furthermore, when acquiring user displacement interactive operation data, a high-precision six-axis sensor can be used, with the sampling frequency set to 100 Hz, and the data accuracy reaches 1° and 0.1 m / s. 2 , and supports Bluetooth low energy 0 protocol to achieve low-latency data transmission. Kalman filtering is performed on the collected data, and the noise covariance matrix is ​​adaptively adjusted to effectively remove high-frequency noise and outliers, and to extract smooth displacement and rotation curves. The density-based DBSCAN clustering algorithm is adopted, and the density radius is set to 5 and the minimum number of points is 10. It can find user groups with similar operating habits. Then, the C5 decision tree algorithm is used to select split attributes through the information gain ratio to generate a decision tree with an average depth of 5, and the accuracy rate reaches more than 95%. In real-time interaction, according to the user's operation data, the decision tree is traversed within 10ms, and parameters such as joystick sensitivity and gesture recognition threshold are adjusted to achieve smooth and natural interactive response. At the same time, 1GB of user log data is analyzed regularly every day, and the clustering and decision models are updated to adapt to the changing trend of user operation habits. Finally, according to the user's interaction preferences, the collaborative filtering algorithm is used to recommend 10 most matching solutions from 1,000 preset interaction configurations to meet the personalized needs of different users and improve the user's interactive experience satisfaction.

[0052] Embodiment 2:

[0053] The embodiment of the present invention further provides an ancient building protection and display system based on virtual reality, comprising:

[0054] A first processing device, used to map the collected user position coordinates and orientation angle data to corresponding positions in the virtual scene, and dynamically update the viewing angle in the virtual scene according to the user's movement and turning;

[0055] A second processing device is used to pre-process the virtual scene to obtain a spatial division result of dividing the virtual scene into a plurality of small areas;

[0056] The third processing device is used to predict and compensate for the displacement and rotational motion of the user, fuse and optimize the sensor data, and obtain the position and direction estimation results with less noise interference;

[0057] The fourth processing device is used to make a detailed geometric description and spatial division of buildings and obstacles in the virtual scene, perform ray detection in real time during the user's displacement, and restrict or guide the user's movement based on the detection results.

[0058] As an implementation method of an embodiment of the present invention, the second processing device uses an octree space division algorithm to pre-process the virtual scene, obtains a spatial division result of dividing the virtual scene into multiple small areas, and establishes a bounding box for each small area. When the user performs displacement interaction, the small area where the user's current position is located is determined, and only collision detection and rendering are performed on virtual objects within the small area.

[0059] As an implementation method of an embodiment of the present invention, the third processing device uses an inertial navigation algorithm and data collected by an accelerometer and a gyroscope to predict and compensate for the user's displacement and rotational movement, and uses a Kalman filtering algorithm to fuse and optimize the sensor data to obtain position and direction estimation results with less noise interference.

[0060] As an implementation mode of an embodiment of the present invention, the fourth processing device performs a detailed geometric description and spatial division of buildings and obstacles in the virtual scene based on a high-precision three-dimensional collision detection model, performs ray detection in real time during the user's displacement, determines the distance and collision relationship between the user and virtual objects in the virtual scene, and restricts or guides the user's movement based on the detection results.

[0061] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for protecting and displaying ancient buildings based on virtual reality, characterized in that: include: Step S1, mapping the collected user position coordinates and orientation angle data to the corresponding position of the virtual scene, and dynamically updating the viewing angle in the virtual scene according to the user's movement and turning; Step S2, preprocessing the virtual scene to obtain a spatial division result of dividing the virtual scene into multiple small areas; Step S3: predict and compensate for the user's displacement and rotational motion, fuse and optimize the sensor data, and obtain position and direction estimation results with less noise interference; Step S4: Perform detailed geometric description and spatial division of buildings and obstacles in the virtual scene, perform ray detection in real time during the user's displacement, and restrict or guide the user's movement based on the detection results.

2. The ancient building protection and display method based on virtual reality according to claim 1 is characterized in that: In step S2, the virtual scene is preprocessed using an octree space division algorithm to obtain a space division result that divides the virtual scene into multiple small areas, and a bounding box is established for each small area. When the user performs displacement interaction, the small area where the user's current position is located is determined, and collision detection and rendering are performed only on virtual objects within the small area.

3. The ancient building protection and display method based on virtual reality as claimed in claim 2 is characterized in that: In step S3, the inertial navigation algorithm uses the data collected by the accelerometer and gyroscope to predict and compensate for the user's displacement and rotational movement, and the Kalman filter algorithm is used to fuse and optimize the sensor data to obtain position and direction estimation results with less noise interference.

4. The ancient building protection and display method based on virtual reality as claimed in claim 3 is characterized in that: In step S4, based on the high-precision three-dimensional collision detection model, the buildings and obstacles in the virtual scene are finely geometrically described and spatially divided. During the user's displacement, ray detection is performed in real time to determine the distance and collision relationship between the user and the virtual objects in the virtual scene, and the user's movement is restricted or guided based on the detection results.

5. A virtual reality-based ancient building protection and display system, characterized in that: include: A first processing device, used to map the collected user position coordinates and orientation angle data to corresponding positions in the virtual scene, and dynamically update the viewing angle in the virtual scene according to the user's movement and turning; A second processing device is used to pre-process the virtual scene to obtain a spatial division result of dividing the virtual scene into a plurality of small areas; The third processing device is used to predict and compensate for the displacement and rotational motion of the user, fuse and optimize the sensor data, and obtain the position and direction estimation results with less noise interference; The fourth processing device is used to make a detailed geometric description and spatial division of buildings and obstacles in the virtual scene, perform ray detection in real time during the user's displacement, and restrict or guide the user's movement based on the detection results.

6. The ancient building protection and display system based on virtual reality according to claim 5 is characterized in that: The second processing device uses an octree space division algorithm to pre-process the virtual scene, obtains a space division result that divides the virtual scene into multiple small areas, and establishes a bounding box for each small area. When the user performs displacement interaction, the small area where the user's current position is located is determined, and collision detection and rendering are only performed on virtual objects within the small area.

7. The ancient building protection and display system based on virtual reality according to claim 6 is characterized in that: The third processing device predicts and compensates for the user's displacement and rotational movement through an inertial navigation algorithm and the data collected by the accelerometer and gyroscope, and uses a Kalman filter algorithm to fuse and optimize the sensor data to obtain position and direction estimation results with less noise interference.

8. The ancient building protection and display system based on virtual reality according to claim 7 is characterized in that: The fourth processing device performs detailed geometric description and spatial division of buildings and obstacles in the virtual scene based on the high-precision three-dimensional collision detection model, performs ray detection in real time during the user's displacement, determines the distance and collision relationship between the user and virtual objects in the virtual scene, and restricts or guides the user's movement based on the detection results.

Images