A virtual reality-based ancient building protection display method and system

Abstract

The application discloses a kind of ancient building protection display method and system based on virtual reality, comprising: obtaining the position coordinates and orientation angle data of user position that high-precision position tracking sensor real-time acquisition, the data is mapped to the corresponding position of virtual scene, according to the dynamic updating of the perspective in the virtual scene of user's movement and steering, immersive experience is realized in situ;Adopt the rendering technique based on physics, the light, material, texture etc. of the virtual scene are simulated realistically, through pre-computed light map, environmental occlusion etc. technique, reduce real-time rendering calculation amount, improve the rendering efficiency and visual quality of the virtual scene;Design multiple displacement interaction modes, including traditional joystick movement, gesture control, body inclination etc., meet different user operation habits, analyze and model user interaction behavior through machine learning algorithm, adaptively adjust interaction parameters and response mode, provide personalized displacement interaction experience.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of information technology, and particularly relates to a virtual reality-based ancient building protection display method and system. BACKGROUND

[0002] In the virtual reality-based ancient building protection display system, displacement technology is the key to realizing the free exploration of users in the virtual scene. However, there are some technical difficulties and contradictions in the displacement of the virtual scene. First, in order to obtain an immersive experience, the system needs to capture the position and direction changes of the user in real time and accurately, and map them to the virtual scene. This puts higher requirements on the sensitivity and response speed of the sensor. Second, displacement boundary and collision detection in the virtual scene is also a big challenge. In the real world, people can judge the position and distance of objects through visual and tactile senses to avoid collision. However, in the virtual scene, due to the lack of real tactile feedback, users may misjudge the distance, resulting in crossing objects or crossing boundaries. In addition, the virtual scene in the ancient building display system usually contains a large number of fine architectural details and decorative elements. How to ensure the fidelity while optimizing the rendering efficiency of the scene, reducing the lag and delay in the displacement process, is also a problem to be solved. Finally, different users have different heights, body shapes and behavior habits. How to design a displacement interaction mode that is adaptable, flexible and variable to meet the needs of different users is also a topic worth exploring. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a virtual reality-based ancient building protection display method and system.

[0004] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0005] A virtual reality-based ancient building protection display method, comprising:

[0006] Step S1, mapping the collected user position coordinates and orientation angle data to the corresponding positions of the virtual scene, and dynamically updating the view angle in the virtual scene according to the movement and turning of the user;

[0007] Step S2, preprocessing the virtual scene to obtain a spatial division result dividing the virtual scene into multiple small areas;

[0008] Step S3, predicting and compensating the displacement and rotation motion of the user, fusing and optimizing the sensor data to obtain a position and direction estimation result with less noise interference;

[0009] Step S4, fine geometry description and space division are performed on the buildings and obstacles in the virtual scene, and ray detection is performed in real time during user displacement, and the movement of the user is limited or guided according to the detection result.

[0010] Preferably, in step S2, the virtual scene is preprocessed by using an octree space division algorithm to obtain a space division result of dividing the virtual scene into a plurality of small regions, and a bounding box is established for each small region, and when the user performs displacement interaction, the small region where the current position of the user is located is judged, and only the virtual objects in the small region are subjected to collision detection and rendering.

[0011] Preferably, in step S3, the displacement and rotation movement of the user are predicted and compensated by using the data collected by the accelerometer and the gyroscope through an inertial navigation algorithm, the sensor data is fused and optimized by using a Kalman filtering algorithm, and a position and direction estimation result with less noise interference is obtained.

[0012] Preferably, in step S4, fine geometry description and space division are performed on the buildings and obstacles in the virtual scene according to a high-precision three-dimensional collision detection model, ray detection is performed in real time during user displacement, the distance and collision relationship between the user and the virtual objects in the virtual scene are judged, and the movement of the user is limited or guided according to the detection result.

[0013] The application also provides an ancient building protection display system based on virtual reality, comprising:

[0014] The first processing device is used for mapping the collected user position coordinate and orientation angle data to the corresponding position of the virtual scene, and dynamically updating the view angle in the virtual scene according to the movement and turning of the user;

[0015] The second processing device is used for preprocessing the virtual scene to obtain a space division result of dividing the virtual scene into a plurality of small regions;

[0016] The third processing device is used for predicting and compensating the displacement and rotation movement of the user, fusing and optimizing the sensor data, and obtaining a position and direction estimation result with less noise interference;

[0017] The fourth processing device is used for fine geometry description and space division on the buildings and obstacles in the virtual scene, real-time ray detection during user displacement, and limiting or guiding the movement of the user according to the detection result.

[0018] Preferably, the second processing device adopts an octree space partitioning algorithm to pre-process the virtual scene, obtains a space partitioning result of partitioning the virtual scene into a plurality of small regions, and establishes a bounding box for each small region, and when the user performs displacement interaction, judges the small region where the current position of the user is located, and only performs collision detection and rendering on virtual objects in the small region.

[0019] Preferably, the third processing device predicts and compensates the displacement and rotation movement of the user by using data collected by an accelerometer and a gyroscope through an inertial navigation algorithm, and adopts a Kalman filtering algorithm to fuse and optimize the sensor data, so as to obtain a position and direction estimation result with less noise interference.

[0020] Preferably, the fourth processing device performs fine geometric description and space partitioning on buildings and obstacles in the virtual scene according to a high-precision three-dimensional collision detection model, performs real-time ray detection during the displacement of the user, judges the distance and collision relationship between the user and virtual objects in the virtual scene, and limits or guides the movement of the user according to the detection result.

[0021] The application realizes dynamic view angle updating by collecting user position and orientation data in real time through a high-precision position tracking sensor and mapping them to a virtual scene. In order to improve rendering efficiency, the application adopts an octree space partitioning algorithm to pre-process the virtual scene, and combines an inertial navigation algorithm and a Kalman filtering algorithm to optimize position estimation. At the same time, a high-precision three-dimensional collision detection model is constructed to perform real-time ray detection to judge the distance and collision relationship between the user and virtual objects. In terms of rendering, the application adopts a physics-based technology, combined with methods such as pre-computed light maps to improve visual quality. In addition, the application also designs a plurality of displacement interaction modes, and adjusts interaction parameters adaptively through a machine learning algorithm to provide personalized experience for users. The comprehensive application of these technologies significantly improves the immersion and interaction effect of the virtual scene, and brings users a more realistic and smooth virtual experience. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0023] Figure 1 Flow chart of the virtual reality ancient building protection display method of the present application. DETAILED DESCRIPTION

[0024] Clearly, the embodiments described are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0025] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0026] Embodiment 1:

[0027] As shown in the drawings, the embodiment of the present application provides a virtual reality-based ancient building protection display method, comprising: Figure 1

[0028] S101, acquiring user position coordinates and orientation angle data collected by a high-precision position tracking sensor in real time, mapping the data to the corresponding position of a virtual scene, dynamically updating the view angle in the virtual scene according to the movement and turning of the user, and realizing immersive experience.

[0029] Acquiring user position coordinates and orientation angle data collected by a high-precision position tracking sensor in real time, mapping the acquired position coordinate data to the corresponding three-dimensional space position in the virtual scene through coordinate conversion. According to the orientation angle data of the user, the view angle direction in the virtual scene is determined, and the view angle in the virtual scene is synchronized with the actual orientation of the user. The position coordinate change of the user is tracked in real time, and the view angle position in the virtual scene is dynamically adjusted according to the change amplitude and direction of the position coordinates, so as to realize the synchronous update of the virtual scene and the movement of the user. The angle change data of the user turning action is acquired, and the view angle direction of the virtual scene is dynamically adjusted according to the change amplitude and direction of the angle, so as to realize the real-time synchronization of the view angle direction in the virtual scene and the user turning. The view angle position and direction in the virtual scene are synchronized with the actual position and direction of the user in real time, a three-dimensional virtual scene image consistent with the view angle of the user is generated through a rendering engine, and an immersive visual experience is provided. The environment sound and sound in the virtual scene are rendered through an audio engine, the sound source position and volume are dynamically adjusted according to the change of the user position and orientation, a realistic sound effect is created, and the sense of immersion is improved. The visual rendering and audio rendering of the virtual scene are integrated to provide the user with an immersive experience, so that the user can obtain a highly consistent sensory experience in the virtual scene as in the real scene, and the sense of substitution and the sense of immersion of the user are improved.

[0030] ​Further, the high-precision position tracking sensor adopts a combination of an inertial measurement unit (IMU) and visual odometry (Visual Odometry) to collect the position coordinate (x, y, z) and orientation angle (yaw, pitch, roll) data of the user in real time at a frequency of 100 Hz. By multiplying the obtained position coordinate data by the scaling factor 5 of the virtual scene and adding the virtual scene origin offset (1000, 1000, 0), the data is mapped to the corresponding three-dimensional space position in the virtual scene. According to the user's orientation angle data, the viewing angle orientation in the virtual scene is determined to be consistent with it through rotation matrix transformation. The system tracks the user's position coordinate changes in real time, and when the position coordinate changes exceed the threshold of 10 cm or the angle between the change direction and the last change direction exceeds 30°, the dynamic adjustment of the viewing angle position in the virtual scene is triggered, and the update is synchronized at a refresh rate of 60 frames per second. When the angle change amplitude of the user's turning action is detected to exceed 5°, the viewing angle orientation of the virtual scene is dynamically adjusted according to the direction of the angle change at a rate of 1° per degree, realizing the real-time synchronization of the viewing angle orientation and the user's turning. By synchronizing the viewing angle position and orientation in the virtual scene with the actual position and orientation of the user in real time, a high-quality three-dimensional virtual scene image consistent with the user's viewing angle is generated at a rendering frame rate of 90 frames per second using a physically-based rendering (PBR) engine, providing an immersive visual experience. At the same time, the audio engine uses the head-related transfer function (HRTF) algorithm to perform real-time reverberation and positioning of the environmental sound and sound in the virtual scene according to the changes in the user's position and orientation, dynamically adjusting the sound source position and volume attenuation, creating a realistic three-dimensional stereo sound effect, and improving the spatial sense and orientation sense of the sound. Ultimately, through the seamless combination of visual rendering and audio rendering, the user is provided with a full-range, multi-sensory immersive experience, enabling them to obtain a highly consistent visual and auditory experience in the virtual scene as in the real scene, greatly enhancing the user's sense of immersion and immersion, and bringing a realistic and immersive experience.

[0031] S102, the virtual scene is preprocessed using an octree space partitioning algorithm to obtain a space partitioning result that divides the virtual scene into a plurality of small regions, and a bounding box is established for each small region; when the user performs displacement interaction, the small region in which the user's current position is located is determined, and only the virtual objects in the small region are subjected to collision detection and rendering.

[0032] The octree algorithm is used for spatial division preprocessing of the virtual scene, the virtual scene is divided into multiple small areas, and a bounding box is established for each small area, to obtain the spatial division result and the bounding box information. The virtual scene model data established in advance is obtained, the virtual scene model data is divided into multiple sub-model data according to the spatial division result, and each sub-model data corresponds to a small area. For each small area, the virtual object in the area is determined according to the bounding box information, and the mapping relationship between the virtual object and the bounding box is established. When the user performs displacement interaction, the current position coordinates of the user are obtained in real time, the small area where the coordinates are located is judged, and the current area where the user is located is determined. The virtual object data in the current area where the user is located is obtained, the virtual object range that needs to be subjected to collision detection is quickly locked according to the mapping relationship between the virtual object and the bounding box. The hierarchical bounding box algorithm is used for collision detection of the virtual object determined in step 5, whether the user collides with the virtual object is judged, and the collision detection result is obtained. According to the collision detection result, the frustum culling algorithm and the occlusion culling algorithm are used for rendering processing of the virtual object in the current area where the user is located, and the picture display in the virtual scene is updated in real time.

[0033] Further, the octree algorithm is used for spatial division preprocessing of the virtual scene, the virtual scene is divided into a plurality of small areas, the size of each area is 10m*10m*10m, and a bounding box is established for each small area, the minimum and maximum vertex coordinates of the bounding box are calculated to obtain the spatial division result and the bounding box information. The virtual scene model data established in advance is acquired, the virtual scene model data is divided into a plurality of sub-model data according to the spatial division result, each sub-model data corresponds to a small area, and the sub-model data includes the geometric information, texture information and position information of the virtual object in the area. For each small area, the AABB collision detection algorithm is used according to the bounding box information to determine the virtual object in the area, and a mapping relationship between the virtual object and the bounding box is established, the ID of the virtual object is associated with the ID of the bounding box, so as to quickly search in the collision detection. When the user performs displacement interaction, the current position coordinates of the user are acquired in real time, the small area where the coordinates are located is determined through the comparison of the coordinates and the bounding box, and the current area where the user is located is determined. The virtual object data in the current area where the user is located is acquired, the virtual object range that needs to be subjected to collision detection is quickly locked according to the mapping relationship between the virtual object and the bounding box, and the calculation amount of the collision detection is reduced to 1 / 8 of the original. The hierarchical bounding box algorithm is used to establish a hierarchical bounding box tree for the virtual object that needs to be subjected to collision detection, the collision detection is performed layer by layer from top to bottom through the hierarchical structure of the tree, whether the user collides with the virtual object is judged, and the collision detection result is acquired, the time complexity of the collision detection can be reduced to O (logn). According to the collision detection result, the frustum culling algorithm and the occlusion culling algorithm are used to perform visibility judgment on the virtual object in the current area where the user is located, the objects that are not in the frustum range and are occluded are removed, the number of objects to be rendered is reduced, the rendering efficiency is improved, the picture display in the virtual scene is updated in real time, and the visual experience of the user in the moving process is smooth and not stuck.

[0034] According to the collision detection result, the frustum culling algorithm and the occlusion culling algorithm are used to perform rendering processing on the virtual object in the current area where the user is located, and the picture display in the virtual scene is updated in real time.

[0035] The spatial position information and the visual angle direction information of the current area where the user is located are acquired, and a virtual object range needing to be rendered is determined. For the virtual objects needing to be rendered, a collision detection algorithm is used to determine whether there is a collision between the virtual objects. If there is a collision between the virtual objects, the positions and postures of the virtual objects are adjusted according to the collision detection result, so as to avoid overlapping or penetration phenomenon between the virtual objects. After the collision detection and the virtual object position adjustment are completed, a view frustum culling algorithm is used to determine whether the virtual objects are located within the visual field range of the user. If the virtual objects are located outside the view frustum, the virtual objects are culled from a rendering queue. For the virtual objects located within the view frustum, a blocking culling algorithm is further used to determine whether there is a blocking relationship between the virtual objects. By comparing the distances between the virtual objects and the viewpoint, the virtual objects that are blocked are culled from the rendering queue. According to the results of the view frustum culling and the blocking culling, a final virtual object set needing to be rendered is determined, and the virtual object set is transmitted to a graphics rendering engine for rendering calculation and picture generation. The virtual scene picture generated by the graphics rendering engine is fused with a real environment image to obtain an augmented reality display picture.

[0036] Further, after obtaining the spatial position information and the visual angle direction information of the current area where the user is located, the system can filter out virtual objects within the visual field range of the user as a candidate object set to be rendered by using a view frustum culling algorithm according to a preset visual field range parameter (for example, a visual field angle of 90 degrees and a visual distance of 20 meters). Next, for the candidate object set, the system uses a collision detection algorithm based on hierarchical bounding boxes to quickly determine whether there is a collision between objects by comparing the positional relationship between the bounding boxes of the virtual objects. If it is detected that the bounding boxes of object A and object B intersect, the precise mesh models of the two objects are further compared to calculate the minimum distance between the models. When the minimum distance is less than a preset collision threshold (for example, 0.5 meters), it is determined that the two objects collide. For the virtual objects that collide, the system calculates the collision response according to the physical engine, adjusts the position and posture of the objects by applying a separation force and a friction force, so that they are correctly separated in the next frame of picture, avoiding overlapping or penetration phenomenon. After completing the collision detection and object position adjustment, the system uses an occlusion culling algorithm based on depth buffer to determine the occlusion relationship between the objects by comparing the distance between the virtual objects and the viewpoint. Specifically, the system sorts the objects from front to back according to the distance from the objects to the viewpoint, removes the objects that are far away and occluded from the rendering queue, thereby reducing unnecessary rendering calculation. Finally, the system passes the virtual object set after the removal to the graphics rendering engine, uses GPU acceleration technology to perform rendering operations such as vertex transformation, lighting calculation, and texture mapping, and generates realistic virtual scene pictures. The system then real-time fuses the virtual scene pictures with the real environment images obtained through the camera, uses visual inertial odometry algorithm and the like to realize accurate registration of the virtual objects and the real scene, and finally presents an immersive augmented reality experience.

[0037] In S103, the displacement and rotation movement of the user are predicted and compensated by using data collected by accelerometers, gyroscopes and other sensors through an inertial navigation algorithm; and the sensor data is fused and optimized by using a Kalman filtering algorithm to obtain position and direction estimation results with less noise interference.

[0038] According to the inertial navigation algorithm, the original data collected by sensors such as accelerometers and gyroscopes is obtained. For the obtained original data, a Kalman filter algorithm is used for noise interference elimination processing to obtain optimized sensor data. According to the optimized sensor data, the user's displacement and rotational motion information is obtained through integral operation. If the user's displacement or rotational motion information exceeds the preset threshold, it is judged that the user has corresponding motion, and the direction and amplitude of the motion are obtained. According to the direction and amplitude of the user's motion, the position and direction are predicted by using the inertial navigation algorithm, and the current position coordinates and orientation of the user are obtained. The position coordinates and orientation of the user are taken as input, and data fusion is performed through the Kalman filter algorithm to obtain position and direction estimation results with less noise interference. According to the position and direction estimation results, the user's displacement and rotational motion are compensated to obtain accurate user motion trajectory and direction information.

[0039] Further, in the inertial navigation system, first, the original data is collected by sensors such as accelerometers and gyroscopes, for example, the accelerometer collects three-axis acceleration data every 01 seconds, and the gyroscope collects three-axis angular velocity data every 01 seconds. Then, for the collected original data, a Kalman filter algorithm is used for noise interference elimination processing, by establishing state equation and observation equation, using prior estimation and posterior estimation, continuously updating state variables, obtaining optimized sensor data, for example, after filtering, the noise level of acceleration data is reduced from ±1m / s 2 to ±05m / s 2 . Next, according to the optimized sensor data, the user's displacement and rotation information is obtained through integral operation, for example, by integrating the acceleration data, the user's displacement in three directions is obtained, and by integrating the angular velocity data, the user's rotation angle is obtained. If the user's displacement exceeds 5m or the rotation angle exceeds 30°, it is judged that the user has corresponding motion, and the direction and amplitude of the motion are obtained. According to the direction and amplitude of the user's motion, the position and direction are predicted by using the inertial navigation algorithm, for example, by superimposing the current position coordinates with the displacement to obtain the new position coordinates of the user, and by superimposing the current orientation with the rotation angle to obtain the new orientation of the user. The predicted user position coordinates and orientation are taken as observation values, and the Kalman filter algorithm is used for fusion with prior estimation to obtain position and direction estimation results with less noise interference. Finally, according to the position and direction estimation results, the user's displacement and rotational motion are compensated, for example, when rendering a virtual scene, the position and direction of the virtual camera are adjusted in real time according to the estimated position and orientation, so as to obtain accurate user motion trajectory and direction information, and improve the accuracy and smoothness of virtual reality experience.

[0040] S104, construct a high-precision three-dimensional collision detection model, and finely describe and spatially divide the buildings, obstacles and the like in the virtual scene; in the process of displacement of the user, real-time ray detection is performed to determine the distance and collision relationship between the user and the virtual objects in the virtual scene, and the movement of the user is limited or guided according to the detection result.

[0041] According to the pre-established high-precision three-dimensional virtual scene model, the fine geometric description data and the spatial division data of the buildings and obstacles in the model are obtained; when it is detected that the user has displaced, the three-dimensional coordinates of the user's position in the three-dimensional virtual scene model are mapped out according to the current position information of the user; starting from the user's position coordinates, the virtual objects in the line-of-sight direction of the user are detected in the three-dimensional virtual scene model in real time by using a ray detection algorithm; the result data of the ray detection is obtained to determine whether the distance between the user and the detected virtual object is less than a preset collision distance threshold; if so, it is determined that the user has collided with the virtual object, the movement direction and speed of the user are limited according to the collision relationship data, and the user is prevented from passing through the virtual object; if not, the spatial position data of the virtual object is obtained, the movement direction of the user is guided according to the preset guidance rule, and the user is guided to the target virtual object; according to the results of the user movement limitation and guidance, the position coordinates of the user in the three-dimensional virtual scene model are updated, and the updated position information is fed back to the user to provide an immersive interactive experience.

[0042] Further, first, a three-dimensional modeling software such as 3ds Max or Maya is used to model the target scene with high precision, fine geometric description data of the buildings and obstacles in the scene is obtained, and an octree algorithm is used to spatially divide the scene to generate spatial index data of the scene. When the system detects that the user's position has changed, the current three-dimensional coordinates of the user are calculated by an inertial navigation algorithm and are mapped into the virtual scene model. The system starts from the user's position coordinates, uses a ray casting algorithm to detect virtual objects in the line-of-sight direction of the user in real time at a frequency of 60 frames per second, and calculates the distance between the objects and the user. If the distance is less than a collision threshold of 5 meters, a collision is determined to have occurred, the speed and direction of the user after the collision are calculated according to the material properties (such as hardness, elasticity, etc.) of the virtual object by using a physical engine such as PhysX, the user is limited to avoid passing through the model. If no collision has occurred, the movement direction of the user is guided to the target object according to a preset guidance rule (such as always keeping the target object in the center of the field of view and the distance to the target object being within 2 meters, etc.) by using a potential field method. Finally, the user's position is updated by using a Kalman filtering algorithm according to the results of the user movement limitation and guidance, and the updated position information is fed back to the user at a frequency of 30 frames per second, and an immersive interactive experience is provided through multi-channel feedback such as vision, hearing and touch.

[0043] S105, adopt a physical-based rendering technique to simulate the lighting, material, texture, etc. of the virtual scene; by pre-computing lighting map, environment occlusion, etc. technology, reduce the real-time rendering calculation amount, improve the rendering efficiency and visual quality of the virtual scene.

[0044] According to the geometry model and material properties of the virtual scene, a physical-based rendering algorithm such as path tracing, photon mapping, etc. is adopted to simulate the propagation and reflection of light in the scene, and a realistic lighting effect is obtained. For each material in the scene, according to its physical properties such as roughness, metallicity, etc. parameters, a micro-surface model such as Cook-Torrance model is adopted to simulate the reflection and refraction of light on the material, and a real material appearance is obtained. For the texture in the scene, anisotropic filtering, mipmap, etc. technology is adopted to sample the texture at different viewing angles and distances, avoid aliasing and noise, and improve the clarity and realism of the texture. For the static objects in the scene, pre-compute their lighting map under different lighting conditions, directly sample the map during real-time rendering, avoid repeated calculation, and improve the rendering efficiency. In the pre-computation stage of the scene, the environment occlusion technology is adopted to calculate the occlusion of each shading point by the surrounding geometry, which is used as an additional occlusion factor during real-time rendering, to enhance the spatial sense and realism of the scene. According to the complexity and real-time requirements of the virtual scene, level-of-detail technology is adopted to render models and maps of different precision at different viewing angles and distances, to ensure visual quality while improving rendering efficiency. Optimize the rendering process, adopt multi-thread, GPU parallel computing, etc. technology, fully utilize hardware resources, reduce rendering time, and ensure the real-time interaction performance of the virtual scene.

[0045] Further, in the virtual scene rendering, a path tracing algorithm is adopted to simulate the propagation of light. For different materials, such as metal materials, a Cook-Torrance micro-surface model is adopted, with a roughness parameter of 2 and a metallicity parameter of 8, to simulate the reflection and refraction of light on the metal surface. For texture filtering of anisotropic materials such as wood, an anisotropic filtering algorithm is adopted, and an 8-level texture pyramid is generated using the mipmap technique, with the appropriate texture resolution being dynamically selected according to the viewing angle and distance to ensure efficient texture sampling and avoid distortion. For static objects in the scene, such as buildings, their light maps under 5 different lighting conditions are pre-calculated, and a spherical harmonic function compressed light map is used in real-time rendering, reducing memory usage by 80%. In the pre-computation stage, an environment occlusion technique is used to calculate the occlusion factor of each shading point with 64 directional ray sampling, which is used as an additional occlusion factor in real-time rendering and mixed into the shading result to enhance the spatial sense of the scene. According to the complexity of the scene, a level-of-detail technique is used to divide the scene into 8 levels of detail, and the corresponding models and maps are rendered at different viewing angles, with the rendering efficiency improved by 5 times without compromising visual quality. In the rendering process optimization, a multi-thread parallel computing technique is adopted to parallelize the rendering pipeline on an 8-core CPU, reducing the rendering time by 70% and ensuring real-time interaction performance of the virtual scene.

[0046] In the pre-computation stage of the scene, an environment occlusion technique is used to calculate the occlusion of each shading point by surrounding geometry, which is used as an additional occlusion factor in real-time rendering to enhance the spatial sense and realism of the scene.

[0047] According to the pre-established scene model, the position information of each shading point and the position information of the surrounding geometry in the scene are obtained. For each shading point, the occlusion relationship between it and the surrounding geometry is calculated to obtain the occlusion factor of each shading point. The calculated occlusion factor of each shading point is stored in a pre-set data structure and associated with the corresponding shading point. When rendering the scene in real time, the position information of each shading point in the current rendering frame is obtained. For each shading point, its corresponding occlusion factor is obtained from the pre-set data structure. According to the obtained occlusion factor, the shading calculation of each shading point is adjusted, and the lighting intensity is multiplied by the occlusion factor to simulate the effect of environmental occlusion. The adjusted shading result is applied to the current rendering frame to update the rendering effect of the scene and enhance the spatial sense and realism of the scene.

[0048] Further, in the scene pre-computation stage, a large number of light rays are projected from each shading point to the surrounding using light ray projection and other techniques, and the occlusion relationship is calculated by intersecting the scene geometry. For example, 1024 light rays can be projected at each shading point, distributed on a hemisphere centered at the point, to obtain the intersection information of the light rays with the surrounding geometry. The closer the intersection point is to the shading point, the greater the occlusion factor in the corresponding direction. The calculated occlusion factor is stored in a texture map, with each pixel corresponding to a shading point, and the pixel value being the occlusion factor of the point. Considering the spatial correlation of the occlusion information, the occlusion map can be mipmap processed to store occlusion information of different accuracy at different mipmap levels. During real-time rendering, the occlusion factor is sampled from the pre-computed occlusion map by trilinear interpolation according to the uv coordinates of the current shading point and its depth value in the camera space. For example, for a shading point with uv coordinates (3, 8) and a depth value of 5, interpolation is performed on the adjacent 4 pixels in the 3*width row and 8*height column of the occlusion map, and the depth value is interpolated in the adjacent two mipmap levels of the occlusion map, to finally obtain the occlusion factor of the point. Multiply the sampled occlusion factor by the illumination intensity of the point to obtain the final shading result, which can simulate the real environmental occlusion effect. For example, for a shading point with an occlusion factor of 8, its final illumination intensity is 8 times the original illumination intensity, and the point will show a dark effect of being occluded by surrounding objects during rendering, improving the spatial sense of the scene. By applying shading calculation based on pre-computed occlusion information to all shading points in the scene, a virtual scene picture with real environmental occlusion is finally rendered.

[0049] S106, design multiple displacement interaction modes, including traditional joystick movement, gesture control, body tilt, etc., to meet different user operation habits; analyze and model user interaction behavior through machine learning algorithm, and adaptively adjust interaction parameters and response mode to provide personalized displacement interaction experience.

[0050] The displacement interaction operation data of the user is acquired, including operation parameters in various interaction modes such as joystick movement, gesture control and body tilt. The acquired displacement interaction operation data of the user is preprocessed to remove noise and abnormal data and extract key feature parameters. The clustering algorithm is used to analyze the displacement interaction operation data of the user, and the user is divided into different groups according to operation habits and preferences. For each user group, the displacement interaction response model is established using the decision tree algorithm to determine the optimal interaction parameters and response mode in different interaction modes. When the user performs displacement interaction operation, the operation data is acquired in real time, and the interaction parameters and response mode are dynamically adjusted according to the interaction response model of the user group to which the user belongs. The displacement interaction operation data of the user is continuously collected, and the user group division and interaction response model are regularly updated to adapt to the change of user operation habits. Personalized displacement interaction experience is provided for each user, and the most suitable interaction mode and parameter setting are recommended according to the operation habits and preferences of the user to improve the interaction efficiency and user satisfaction.

[0051] Further, when acquiring the displacement interaction operation data of the user, a high-precision six-axis sensor can be used, the sampling frequency is set to 100Hz, and the data accuracy reaches 1° and 01m / s 2 Meanwhile, Bluetooth Low Energy 0 protocol is supported to realize low-delay data transmission. The collected data is subjected to Kalman filtering, the noise covariance matrix is adaptively adjusted, high-frequency noise and abnormal points are effectively removed, and smooth displacement and rotation curves are extracted. The DBSCAN clustering algorithm based on density is used, the density radius is set to 5, the minimum point number is set to 10, the user groups with similar operation habits can be found, and then the C5 decision tree algorithm is used to select the split attribute through information gain ratio, generate a decision tree with an average depth of 5, and the accuracy is more than 95%. During real-time interaction, the decision tree is traversed within 10ms according to the operation data of the user, the parameters such as joystick sensitivity and gesture recognition threshold are adjusted, and smooth and natural interaction response is realized. Meanwhile, 1GB of user log data is analyzed regularly every day, the clustering and decision models are updated, and the change trend of user operation habits is adapted. Finally, according to the interaction preferences of the user, 10 most matched schemes are recommended from 1000 preset interaction configurations by using the collaborative filtering algorithm, the individual needs of different users are met, and the interaction experience satisfaction of the user is improved.

[0052] Embodiment 2:

[0053] The embodiment of the application also provides a virtual reality-based ancient building protection display system, comprising:

[0054] The first processing device is used for mapping the collected user position coordinate and orientation angle data to the corresponding position of the virtual scene, and dynamically updating the view angle in the virtual scene according to the movement and turning of the user;

[0055] The second processing device is configured to pre-process the virtual scene to obtain a spatial division result of dividing the virtual scene into a plurality of small areas.

[0056] The third processing device is configured to predict and compensate the displacement and rotation movement of the user, fuse and optimize the sensor data, and obtain a position and direction estimation result with less noise interference.

[0057] The fourth processing device is configured to finely describe and spatially divide the buildings and obstacles in the virtual scene, and perform real-time ray detection during the displacement of the user, and limit or guide the movement of the user according to the detection result.

[0058] As an embodiment of the present application, the second processing device adopts an octree spatial division algorithm to pre-process the virtual scene to obtain a spatial division result of dividing the virtual scene into a plurality of small areas, and establishes a bounding box for each small area. When the user performs displacement interaction, the small area where the current position of the user is located is determined, and only the virtual objects in the small area are subjected to collision detection and rendering.

[0059] As an embodiment of the present application, the third processing device uses the data collected by the accelerometer and gyroscope to predict and compensate the displacement and rotation movement of the user through an inertial navigation algorithm, and uses a Kalman filtering algorithm to fuse and optimize the sensor data to obtain a position and direction estimation result with less noise interference.

[0060] As an embodiment of the present application, the fourth processing device finely describes and spatially divides the buildings and obstacles in the virtual scene according to a high-precision three-dimensional collision detection model, performs real-time ray detection during the displacement of the user, determines the distance and collision relationship between the user and the virtual objects in the virtual scene, and limits or guides the movement of the user according to the detection result.

[0061] The above-described embodiments are only descriptions of the preferred modes of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by those skilled in the art should fall within the protection scope of the claims of the present application.

Claims

1. A virtual reality-based ancient building protection display method, characterized in that, Comprising: Step S101, mapping the collected user position coordinates and orientation angle data to the corresponding positions in the virtual scene, dynamically updating the view angle in the virtual scene according to the user's movement and turning; Step S102, using an octree space division algorithm to preprocess the virtual scene, obtaining a space division result that divides the virtual scene into multiple small regions, and establishing a bounding box for each small region. When the user performs displacement interaction, the small region where the user's current position is located is determined, and only the virtual objects in this small region are subjected to collision detection and rendering; Step S103, by using an inertial navigation algorithm, using the data collected by the accelerometer and gyroscope to predict and compensate the displacement and rotation motion of the user, using a Kalman filter algorithm to fuse and optimize the sensor data, and obtaining a position and direction estimation result with less noise interference; Step S104, according to a high-precision three-dimensional collision detection model, finely describing and dividing the buildings and obstacles in the virtual scene, and in the process of user displacement, performing real-time ray detection to determine the distance and collision relationship between the user and the virtual objects in the virtual scene, and limiting or guiding the user's movement according to the detection result; Step 105, according to the geometric model and material properties of the virtual scene, using path tracing and photon mapping rendering algorithms to simulate the propagation and reflection of light in the scene, and obtaining realistic lighting effects; In the pre-computation stage of the scene, use environment occlusion technology to calculate the occlusion of each shading point by surrounding geometry, and use it as an additional occlusion factor in real-time rendering; At the same time, use ray casting technology to project a large number of rays from each shading point to the surrounding, and calculate the occlusion relationship with the scene geometry; Step 106, design multiple displacement interaction modes, including traditional joystick movement, gesture control, body tilt, to meet the operation habits of different users; At the same time, analyze and model the user's interaction behavior, and adaptively adjust the interaction parameters and response mode to provide personalized displacement interaction experience; Specifically: obtain the user's displacement interaction operation data, including operation parameters in multiple interaction modes such as joystick movement, gesture control and body tilt; Preprocess the obtained user displacement interaction operation data to remove noise and abnormal data and extract key feature parameters; Use clustering algorithm to analyze the user's displacement interaction operation data, and divide the users into different groups according to operation habits and preferences; For each user group, use decision tree algorithm to establish displacement interaction response model, determine the optimal interaction parameters and response mode under different interaction modes; When the user performs displacement interaction operation, the operation data is obtained in real time, and the interaction parameters and response mode are dynamically adjusted according to the interaction response model of the user group to which the user belongs; Continuously collect the user's displacement interaction operation data, and regularly update the user group division and interaction response model to adapt to the change of user operation habits.

2. A virtual reality-based ancient building protection display system implementing the virtual reality-based ancient building protection display method of claim 1, characterized in that, Comprising: A first processing device for mapping the collected user position coordinates and orientation angle data to the corresponding positions in the virtual scene, and dynamically updating the view angle in the virtual scene according to the user's movement and turning; The second processing device is used for preprocessing the virtual scene to obtain a space division result of dividing the virtual scene into a plurality of small areas; The third processing device is used for predicting and compensating the displacement and rotation movement of the user, fusing and optimizing the sensor data to obtain a position and direction estimation result with less noise interference; The fourth processing device is used for performing fine geometric description and space division on the buildings and obstacles in the virtual scene, performing real-time ray detection during the displacement of the user, and limiting or guiding the movement of the user according to the detection result.

Images