Centimeter-level VR real-scene real-time synchronous rendering method and system for multi-source data fusion and physical field coupling

Abstract

The invention relates to the technical field of virtual reality, in particular to a centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling. Comprising the following steps: eliminating cloud layer interference and expanding Kalman filtering to improve positioning precision through multi-source acquisition of satellite remote sensing data, meteorological data and an equipment sensor; dynamically adjusting process noise by using Kalman filtering, and complementing delay data; constructing a ray tracing and U-Net noise reduction model, carrying out hierarchical loading of textures in combination with dynamic texture management, modeling dynamic illumination and wind field simulation based on a solar declination angle, and realizing scene rendering; a virtual ground is aligned through WebXR, gesture recognition and a force feedback mechanism are integrated, and fingertip curvature and movement speed features are extracted in combination with a Leap Motion sensor. According to the invention, through multi-source data fusion, efficient rendering and accurate interactive design, the authenticity and real-time synchronization capability of the VR scene are significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and particularly to a centimeter-level VR real-scene real-time synchronous rendering method and system for multi-source data fusion and physical field coupling. Background Art

[0002] Traditional virtual reality systems have certain deficiencies due to being limited by static data sources and low-resolution geographical information: the lack of dynamic environmental elements leads to the inability to synchronize real-world weather, light, and humidity changes in real time, resulting in insufficient scene immersion; the data update mechanism is lagging, for example, Google Earth VR relies on periodic satellite image updates and is difficult to meet the demand for second-level dynamic response; the computing efficiency is low. Especially when using a generative adversarial network (GAN) for weather simulation, the huge amount of calculations severely restricts the real-time rendering performance; the static 3D modeling tools (Blender / 3ds Max) in the existing technology system lack the ability to adaptively update the environment, there are significant data delays in the combined solutions of geographical information systems and VR, and the artificial intelligence enhanced rendering solutions are difficult to maintain high-frame-rate dynamic rendering quality due to hardware computing power limitations. Summary of the Invention

[0003] The object of the present invention is to propose a centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling in view of the problems existing in the background art.

[0004] The technical solution of the present invention: A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling includes the following specific implementation steps:

[0005] S1. Obtain satellite remote sensing data and real-time synchronous meteorological data, and fuse IMU data and GPS data through an extended Kalman filter;

[0006] S2. Use a Kalman filter to smooth the time sequence difference of multi-source data, complete satellite data with a delay exceeding 5 seconds through a cubic spline interpolation method, and dynamically adjust the process noise covariance matrix;

[0007] S3. Dynamically calculate solar illumination, including modeling the light source direction based on the solar declination angle and hour angle, adjusting the light intensity attenuation coefficient according to the environmental humidity, and constructing a wind field dynamics simulation. Generate a wind speed field through the Lattice Boltzmann method, i.e., LBM, and drive the scene elements to respond to the wind field changes. Use RTX hardware acceleration for ray tracing and a U-Net denoising model for ray tracing and denoising, and perform dynamic texture management, and simulate the optical effects of clouds;

[0008] S4. Access the virtual environment through a WebXR device, synchronize the user's head pose data in real time with a latency less than 20 ms, and combine gesture recognition and force feedback mechanisms. Extract gesture features based on the Leap Motion sensor and generate collision feedback through the Cannon.js physics engine.

[0009] Preferably, the preprocessing of satellite remote sensing data includes: collecting point cloud data of the target area through a Velodyne VLS-128 lidar; using the iterative closest point algorithm to minimize the spatial deviation between the satellite image and the point cloud, outputting the rotation matrix R and the translation vector t, and iteratively optimizing until convergence to output the calibrated geographical coordinates.

[0010] Preferably, the specific steps for fusing IMU data and GPS data include:

[0011] S31. Predict the system position and velocity through the state equation:

[0012]

[0013] where X k+1 represents the position at time k + 1; x k represents the position at time k; represents the velocity of the system at time k; represents the acceleration at time k; Δt represents the discrete time step; represents the velocity at time k + 1;

[0014] S32. Use the GPS data as the observation value, and set the covariance matrix as R = diag(1.0, 1.0, 1.0);

[0015] where diag() represents the diagonal matrix.

[0016] Preferably, the parameters of the Kalman filter are set as:

[0017] For IMU data, the process noise covariance matrix Q = diag(0.5, 0.5, 0.5, 0.1, 0.1, 0.1);

[0018] For meteorological data, the process noise covariance Q = 0.1;

[0019] When the noise is non-Gaussian, switch to the variational parameter variational Bayesian Kalman filter algorithm VBKF and dynamically adjust the variational parameters;

[0020] where diag() represents the diagonal matrix.

[0021] Preferably, the wind field dynamics simulation includes:

[0022] Three-dimensional fluid simulation is carried out using the LBM and D3Q27 lattice model;

[0023] The collision and flow calculation of the distribution function are realized through WebGL Compute Shader;

[0024] The external force term of the temperature gradient is added to the LBM evolution equation as a source term to drive the dynamic response of the scene elements.

[0025] Preferably, ray tracing and noise reduction include: 64 ray samples are taken per pixel, the recursion depth is 3 times, and a pre-trained model with a U-Net structure is used to reduce the noise of the noisy frame.

[0026] Preferably, the aerosol-humidity coupled lighting model is:

[0027] S71. Dynamic calculation of solar illumination:

[0028] Modeling of the sun's position: Calculation of the declination angle and hour angle:

[0029]

[0030] In the formula, σ represents the solar declination angle; n represents the day of the year; H represents the solar hour angle; h UTC represents Coordinated Universal Time; λ represents the local longitude; E time represents the time difference, the difference between true solar time and mean solar time, which corrects the uneven apparent motion caused by the Earth's orbital revolution and the tilt of the axis of rotation;

[0031] S72. Attenuation of light intensity: Based on the humidity absorption coefficient a = 0.02·(1 + humidity / 100), the intensity of the parallel light is dynamically adjusted:

[0032]

[0033]

[0034] In the formula, I represents the attenuated light intensity; I0 represents the initial light intensity; H' represents the equivalent thickness of the atmosphere; humidity represents the environmental humidity; e represents the natural constant; V represents the environmental visibility; Q txt represents the aerosol extinction coefficient; r represents the radius of the aerosol particles; λ is the light wavelength; m is the complex refractive index of the aerosol;

[0035] S73. When the visibility V < 10 km in the meteorological data, Mie scattering calculation is activated;

[0036] In the fragment shader, according to V and Q ext dynamically adjust the light attenuation intensity to simulate the scattering effect of haze / dust;

[0037] S74. Three.js Light Source Settings: Update the direction and color of the directional light in real time according to the solar azimuth and altitude angles.

[0038] Preferably, the wind field dynamics simulation further includes:

[0039] Introduce the Coriolis force term into the Navier-Stokes equation

[0040] where Ω is the angular velocity vector of the Earth's rotation;

[0041] Dynamically adjust the components of Ω according to the latitude φ where the user is located, and the direction is determined by the right-hand rule;

[0042] Explicitly discretize the Coriolis force term by the finite difference method.

[0043] Preferably, the force feedback mechanism includes: driving the device to vibrate at a frequency of 200 Hz through the WebHaptics API, and the vibration duration is proportional to the magnitude of the virtual collision force.

[0044] The technical solution of the present invention: A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling, which is used to execute the above-mentioned centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling, includes:

[0045] Data acquisition layer, obtaining satellite remote sensing data and real-time meteorological station data;

[0046] Data fusion layer, using Kalman filter to smooth time series data and using variational Bayesian Kalman filter (VBKF) to process non-Gaussian noise;

[0047] Rendering engine layer, dynamically calculating solar illumination, adjusting the light intensity attenuation coefficient according to the environmental humidity, constructing a wind field dynamics simulation, generating a wind speed field, driving scene elements to respond to wind field changes, performing ray tracing and noise reduction, performing dynamic texture management, and simulating cloud optical effects;

[0048] VR interaction layer, accelerating fluid simulation to achieve dynamic changes in the wind field.

[0049] Compared with the prior art, the above technical solution of the present invention has the following beneficial technical effects:

[0050] The present invention designs a centimeter-level VR real-scene real-time synchronous rendering method that combines multi-source data fusion and physical field coupling. Through multi-source data fusion, dynamic environment simulation, an efficient rendering engine, and low-latency interaction design, it significantly improves the authenticity, real-time performance, and immersion of VR scenes. By efficiently integrating cross-domain technologies and parametric optimization, it solves problems such as single data, low rendering efficiency, and high interaction latency in existing VR technologies, and has outstanding technical effects and application values:

[0051] (1) Multi-source data fusion enhances scene authenticity: By preferentially selecting Sentinel-2 / Landsat-8 satellite data (with dynamic cloud coverage switching) and real-time synchronous OpenWeather API meteorological data, combined with cloud masking processing and terrain correction algorithms, it ensures that the surface texture, vegetation coverage, and real environment are consistent. And it uses the Extended Kalman Filter (EKF) to fuse IMU (sampled at 100Hz) and GPS data, improving the positioning accuracy from ±1 meter to ±0.3 meter, significantly reducing the position drift problem in VR scenes;

[0052] (2) Dynamic environment simulation enhances real-time performance: Build an aerosol-humidity coupled lighting model, introduce the Mie scattering term into the original humidity attenuation formula, model the light source direction based on the solar declination angle and hour angle, and combine the attenuation coefficient of humidity on light intensity to achieve lighting changes synchronized with the real environment (such as simulating the color temperature at dawn and dusk). And it conducts high-precision simulation of wind field dynamics, adds the Coriolis force term to the Navier-Stokes equation to drive the swaying of trees and the movement of the particle system, and responds to real-time wind speed changes;

[0053] (3) Double optimization of rendering efficiency and quality: Adopt RTX hardware acceleration (64 samples per pixel, recursive depth 3) combined with the U-Net denoising model (inference latency ≤ 10ms). While ensuring the rendering quality (soft shadows, dynamic reflections), it maintains a high frame rate of 90FPS. And it conducts a dynamic texture management strategy, hierarchically loads textures based on the LOD strategy (near view resolution of 1 meter, middle view resolution of 10 meters, far view resolution of 50 meters), and preloads asynchronously through Web Worker to avoid rendering stuttering;

[0054] (4) Low-latency VR interaction and enhanced immersion: Adopt the "local-floor" reference space of WebXR, combined with sensor calibration of the ground height (error < 0.1 meter), eliminate the user's dizziness, and improve the stability of the standing / sitting experience. And it conducts gesture recognition and force feedback linkage, extracts gesture features (such as fingertip curvature, hand speed) through the Leap Motion sensor, combines the Cannon.js physics engine to generate collision feedback, and realizes 200Hz vibration haptics through the WebHaptics API to enhance the interaction realism;

[0055] (5) Improvement in data processing and transmission efficiency: Data preprocessing and compression are carried out. Satellite images are stored in WebP format after compression (compression ratio: 80%). Meteorological data are filtered through a sliding window (window size = 3) and outliers are removed to reduce redundant data transmission. Also, time series consistency is ensured. The process noise covariance matrix is dynamically adjusted using Kalman filtering, and delayed data are complemented through cubic spline interpolation to ensure the spatio-temporal synchronization of multi-source data. Description of the Drawings

[0056] Figure 1 This is a logic diagram of a centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling proposed by the present invention. Detailed Implementation Manner

[0057] Example 1. A centimeter-level VR real-scene real-time synchronous rendering system for multi-source data fusion and physical field coupling proposed by the present invention includes:

[0058] Data acquisition layer: Obtain satellite remote sensing data and real-time data from meteorological stations;

[0059] Data fusion layer: Use Kalman filtering to smooth time series data and use variational Bayesian Kalman filtering (VBKF) to process non-Gaussian noise;

[0060] Rendering engine layer: Dynamically calculate solar illumination, adjust the light intensity attenuation coefficient according to environmental humidity, construct a wind field dynamics simulation, generate a wind speed field, drive scene elements to respond to wind field changes, perform ray tracing and noise reduction, perform dynamic texture management, and simulate cloud optical effects;

[0061] VR interaction layer: Accelerate fluid simulation to achieve dynamic changes in the wind field.

[0062] Example 2. As Figure 1 shown, a centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling proposed by the present invention is applied to a centimeter-level VR real-scene real-time synchronous rendering system for multi-source data fusion and physical field coupling proposed in Example 1. The specific implementation steps are as follows:

[0063] S1. Data acquisition layer: Obtain satellite remote sensing data and real-time data from meteorological stations. Specifically:

[0064] S11. Acquisition and processing of satellite remote sensing data:

[0065] (1) Selection and retrieval of data sources:

[0066] Satellite Selection: Preferentially select L2A level data (data after atmospheric correction) of Sentinel-2 satellites. If the cloud coverage rate of the target area exceeds 20%, switch to the OLI sensor data of Landsat-8;

[0067] Time Window Screening: Obtain images of the target area in the last 5 days through the Google Earth Engine API, and use the median synthesis method (median()) to eliminate cloud interference;

[0068] Band Extraction: Extract visible light (B4, B3, B2), near-infrared (B8), and short-wave infrared (B11) bands for surface texture and vegetation cover analysis;

[0069] (2) Preprocessing Process:

[0070] Cloud Mask Processing: Use the cloud mask markings of the QA60 band to filter cloud pixels and generate cloud-free images;

[0071] Radiometric Correction: Use the terrain correction algorithm to eliminate the influence of terrain shadows and atmospheric scattering;

[0072] Tile Generation: Cut the image into tiles of 256×256 pixels, compress and store them in the WebP format, and set the compression rate to 80%;

[0073] (3) LiDAR Point Cloud Compensation:

[0074] Point Cloud Acquisition and Registration: Scan the target area through a Velodyne VLS-128 LiDAR to generate high-precision point clouds (resolution 0.1°×0.1°, accuracy ±2 cm); and extract point cloud features (including but not limited to building edges, road markings) as ground truth references;

[0075] Iterative Closest Point (ICP) Algorithm Calibration:

[0076] Minimize the spatial deviation between the satellite image and the LiDAR point cloud, and optimize the objective function as:

[0077]

[0078] where R represents a 3×3 rotation matrix, describing the rotational transformation between the satellite image and the real terrain; t represents a 3×1 translation vector, compensating for geographical coordinate offsets; source_cloud i represents the feature point cloud extracted from the i-th satellite image; target_cloud i represents the i-th LiDAR measured point cloud;

[0079] Iteratively optimize until convergence (residual <5 cm), and output the calibrated geographical coordinates;

[0080] S12. Real-time synchronization of meteorological data:

[0081] (1) Data access and parsing:

[0082] API call strategy: Request real-time meteorological data of the target area once per second through the OpenWeather API, and parse the temperature, humidity, wind speed, precipitation probability, and cloud cover parameters in the JSON response;

[0083] Unit conversion: Convert temperature from Kelvin to Celsius, and wind speed from knots to meters per second;

[0084] (2) Data calibration and caching:

[0085] Moving window filtering: Perform moving average on temperature and humidity data within 3 consecutive seconds (window size = 3) to eliminate instantaneous noise;

[0086] Outlier handling: If the temperature exceeds -50°C to 50°C or the humidity exceeds 0 - 100%, trigger the API to re-request and discard the abnormal data;

[0087] S13. Fusion of user device sensor data:

[0088] (1) Data acquisition:

[0089] IMU data: Obtain the three-axis acceleration (a x , a y , a z ) and angular velocity (ω x , ω y , ω z ) through the device accelerometer and gyroscope, with a sampling frequency of 100Hz;

[0090] GPS data: Receive the latitude and longitude coordinates (WGS84 coordinate system) with an accuracy of ±1 meter, and align the timestamp to the millisecond level;

[0091] (2) Extended Kalman filter (EKF):

[0092] State equation:

[0093] Position update equation:

[0094] Velocity update equation:

[0095] Where, X k+1 represents the position of the system at time k + 1 (along the x direction), predicting the current position based on the position, velocity, and acceleration at the previous time; x k represents the position of the system at time k (along the x direction), serving as the initial condition for position prediction; denotes the velocity of the system at time k (in the x - direction). The change in position is calculated by multiplying the velocity by the time step. denotes the acceleration of the system at time k (in the x - direction); Δt represents the discrete time step, i.e., the time interval from k to k + 1. The continuous - time motion model is discretized to calculate the increments of velocity and position. denotes the velocity of the system at time k + 1 (in the x - direction). The velocity at the current time is predicted based on the velocity and acceleration at the previous time.

[0096] (3) Observation equation: The GPS position data is used as the observation value. The covariance matrix is set as R = diag(1.0, 1.0, 1.0) according to the GPS error.

[0097] (4) Output result: The positioning accuracy after fusion is improved to 0.3 meters, and the update frequency is 60 Hz.

[0098] S2. Data fusion layer: Since the update times of different data sources are different, Kalman filtering is used to smooth the time - series data to ensure consistency:

[0099] S21. Parameter setting:

[0100] Process noise covariance (Q): It is dynamically adjusted according to the sensor type (IMU: Q = diag(0.5, 0.5, 0.5, 0.1, 0.1, 0.1), meteorological data: Q = 0.1);

[0101] S22. Timestamp alignment: If the satellite data delay exceeds 5 seconds, the cubic spline interpolation method is used to complete the missing frames;

[0102] Accordingly: Kalman filtering is used to reduce sensor noise, smooth the time - series data, ensure the spatio - temporal consistency of different data sources, and improve the real - time synchronization effect of the VR scene;

[0103] S23. Variational Bayesian Kalman Filter (VBKF) is used to handle non - Gaussian noise, specifically:

[0104] S2301. Posterior distribution optimization, approximating the posterior probability distribution based on variational inference:

[0105] p(x t |z 1:t ) = ∫p(x t |x t-1 )p(z t |x t )p(x t-1 |z 1:t-1 )dx t-1 ;

[0106] Among them, x t represents the system state (position, velocity); z t represents the observed value (GPS data);

[0107] S2302, Stochastic Gradient Variational Inference: By minimizing the KL divergence, iteratively optimize the variational distribution parameters to solve the limitations of traditional Kalman filtering for Gaussian noise assumptions;

[0108] S2303, Noise Modeling: Allow process noise and observation noise to follow non-Gaussian distributions (including but not limited to mixture Gaussian noise) to improve the fusion robustness of IMU / GPS data in dynamic environments;

[0109] S2304, Non-Gaussian Noise Processing Strategy:

[0110] When it is detected that the kurtosis of the IMU accelerometer noise > 3 (non-Gaussian characteristic), switch to the VBKF algorithm;

[0111] Set the number of variational iterations to 5 times, and the learning rate η = 0.01 to ensure the convergence of the posterior distribution;

[0112] Output the fused state estimation error reduced to ±0.2 m (the original scheme was ±0.3 m);

[0113] S3, Rendering Engine Layer:

[0114] S31, Dynamic Calculation of Sunlight:

[0115] (1) Sun Position Modeling: Calculation of Declination Angle and Hour Angle:

[0116]

[0117] In the formula, σ represents the sun declination angle; n represents the day of the year; H represents the sun hour angle; h UTC represents Coordinated Universal Time; λ represents the local longitude; E time represents the time difference, the difference between true solar time and mean solar time, which corrects the uneven apparent motion caused by the Earth's revolution orbit and the tilt of the axis of rotation;

[0118] (2) Light Intensity Attenuation: Based on the humidity absorption coefficient a = 0.02·(1 + humidity / 100), dynamically adjust the intensity of the parallel light:

[0119]

[0120] Wherein, I represents the attenuated light intensity; I0 represents the initial light intensity; H' represents the atmospheric equivalent thickness, i.e., the propagation path length of light in a medium (such as the atmosphere); humidity represents the environmental humidity; e represents the natural constant; V represents the environmental visibility (unit: km), which is obtained in real time through a meteorological data interface; Q txt represents the aerosol extinction coefficient, which is dynamically calculated according to the aerosol type (including but not limited to PM2.5, dust particles); r represents the aerosol particle radius; λ is the light wavelength; m is the complex refractive index of the aerosol;

[0121] When the visibility V in the meteorological data is < 10 km, the Mie scattering calculation is activated;

[0122] In the fragment shader, according to V and Q ext dynamically adjust the light attenuation intensity to simulate the scattering effect of haze / dust;

[0123] Accordingly: By introducing the aerosol Mie scattering term, the authenticity of light simulation in low visibility environments is significantly improved, and the physical rendering ability of all-weather VR scenarios is perfected;

[0124] (3) Three.js light source settings: According to the solar azimuth and altitude angles, the direction and color of the parallel light are updated in real time (the color temperature simulates the dawn and dusk effects);

[0125] Accordingly: Calculate the real-time azimuth and altitude angles of the sun, dynamically adjust the light source direction, make the lighting in the VR scene consistent with the real environment, and improve the realism;

[0126] S32. Wind field dynamics simulation:

[0127] (1) LBM fluid simulation:

[0128] Lattice model selection: The D3Q27 three-dimensional lattice model is adopted, and 27 discrete velocity directions are defined in three-dimensional space, supporting high-precision vortex and turbulence simulations;

[0129] Evolution of the distribution function: The core algorithm of LBM is implemented through a WebGL Compute Shader, including but not limited to the collision and streaming steps;

[0130] (2) Performance optimization:

[0131] Parallel computing optimization: Utilize the parallel computing power of the GPU to distribute the lattice node calculation tasks to multiple threads, reducing the calculation time by 57% (compared with the Navier-Stokes solution);

[0132] Time step adjustment: Set the LBM time step Δt = 0.02 seconds to maintain numerical stability and simulation accuracy;

[0133] (3) Scene element response:

[0134] Tree swaying model: Map the velocity field output by LBM to the vertex shader to drive the displacement of tree vertices (compatible with the original vertex shader);

[0135] Particle system simulation: Dynamically adjust the particle movement direction according to the velocity field, supporting complex fluid interactions of elements such as smoke and dust;

[0136] S33, Ray tracing and noise reduction:

[0137] (1) RTX hardware acceleration:

[0138] Ray sampling strategy: 64 samples per pixel, recursive depth of 3 times, supporting soft shadows and dynamic reflections;

[0139] Noise reduction model: Use a pre-trained model with a U-Net structure. The input is a noisy frame of 1920×1080, and the output is a noise reduction weight matrix, with an inference delay ≤ 10ms;

[0140] (2) Dynamic texture management:

[0141] LOD strategy: Use a 1-meter resolution texture for the close view (<50 meters), switch to a 10-meter resolution for the middle view (50 - 500 meters), and downgrade to 50 meters for the far view (>500 meters);

[0142] Asynchronous loading: Preload texture data through Web Worker, and the main thread rendering frame rate is stable at 90FPS;

[0143] S34, Cloud optical effect simulation:

[0144] Rayleigh scattering calculation: Calculate the light attenuation intensity according to the cloud thickness H' and wavelength λ': I λ' = I0'·e -β'(λ')H” ;

[0145] In the formula, I λ' represents the light intensity after scattering; I0' represents the initial intensity of the incident light, that is, the original intensity of the light before entering the cloud; β'(λ') represents the Rayleigh scattering coefficient;

[0146] Refraction effect: In the fragment shader, dynamically adjust the transparency and color according to the cloud density to simulate the light penetration effect;

[0147] Accordingly: Through fluid simulation, dynamically adjust the wind field effect in the VR scene, so that elements such as trees, clouds, and smoke change with the wind;

[0148] S35, Geometric data adaptive subdivision and clipping:

[0149] (1) Octree space partitioning:

[0150] Node structure definition: Construct an octree node data structure, including geometric error, centroid coordinates, and 8 child nodes;

[0151] Geometric error calculation: Dynamically calculate the geometric error according to the curvature or density of the triangular patches within the node. The error value is higher in high-curvature regions (such as building edges);

[0152] (2) Frustum culling and adaptive subdivision:

[0153] Subdivision condition determination: Based on the camera position and the geometric error of the node, calculate the screen space error (SSE). If it exceeds the threshold (such as 2 pixels), subdivision is triggered;

[0154] Dynamic loading strategy: Only subdivide the nodes located within the frustum, and preload adjacent potentially visible nodes through an asynchronous thread to avoid rendering stuttering.

[0155] S4. VR interaction layer, accelerating fluid simulation and realizing dynamic changes in the wind field, specifically:

[0156] S41. WebXR device access:

[0157] Session configuration:

[0158] Reference space selection: Adopt the "local-floor" mode to ensure that the user's standing position is aligned with the virtual ground, and the positioning error is < 0.1 meter;

[0159] Attitude synchronization: Real-time obtain the user's head attitude (position and quaternion rotation), update the virtual camera matrix, with a delay < 20ms;

[0160] It should be noted that "local-floor" is a reference space type defined in WebXR (Web Extended Reality standard), which is used to determine the ground benchmark of the space where the user is located in virtual reality (VR). Its core function is to align the ground of the virtual environment with the physical ground where the user actually stands, so as to ensure that the user's movement in the virtual space is highly consistent with the real environment;

[0161] S42. Gesture recognition and force feedback:

[0162] (1) Gesture feature extraction:

[0163] Finger bend: Calculate the distance between the fingertip and the palm center through the Leap Motion sensor and normalize it to a value between 0 and 1;

[0164] Movement speed: Calculate the displacement difference of the centroid of the hand between consecutive frames. If the speed exceeds 1 m / s, it is determined as "fast sliding".

[0165] (2) Force feedback mechanism:

[0166] Collision detection: Use the Cannon.js physics engine to calculate the contact force between the virtual hand and the scene objects. The feedback intensity is proportional to the collision speed.

[0167] Tactile vibration: Drive the device to vibrate at a frequency of 200 Hz through the WebHaptics API. The duration is related to the magnitude of the collision force.

[0168] Accordingly: Use the WebXR API to connect the VR device to the virtual environment, obtain the user's head position, and achieve an immersive interaction experience.

[0169] Example 3, A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling proposed by the present invention further includes a wind field dynamics simulation, specifically:

[0170] (1) Modification of the Navier-Stokes equation:

[0171] Coriolis force superposition: On the basis of the original external force term (temperature difference convection force), add the Coriolis force term. The complete momentum equation is expressed as:

[0172]

[0173] f thermal = β·Δt;

[0174]

[0175] In the formula, represents the wind field velocity vector, that is, the direction and rate of air flow; t represents the time variable, describing the dynamic change of the wind field over time; ρ represents the air density, affecting the driving effect of the pressure gradient on the flow; p represents the pressure field, the fluid driving force generated by temperature and density differences; v represents the kinematic viscosity of air, reflecting the hindrance of the internal frictional resistance of the fluid to the velocity change; represents the gradient operator, that is, the rate of change of physical quantities (including but not limited to pressure, velocity) in space; represents the Laplace operator, describing the diffusion effect of the velocity field; f thermal represents the temperature difference convection force; β represents the thermal expansion coefficient, that is, the driving intensity of the air density change caused by the temperature gradient on the flow; Δt represents the temperature difference, describing the temperature gradient between adjacent grids in the target area; f coriolis represents the Coriolis force; Ω represents the angular velocity vector of the Earth's rotation;

[0176] Parameter setting: The direction of the angular velocity Ω of the Earth's rotation is determined by the right-hand rule (vertically upward in the Northern Hemisphere). In grid calculations, the component Ω of Ω is adjusted according to the latitude φ. z = ||Ω}}·sin φ;

[0177] (2) Numerical solution optimization:

[0178] Discretization processing: In the finite difference method, the Coriolis force term is added as an explicit source term to the velocity field update step.

[0179] Stability condition: The time step is maintained at Δt = 0.01 s to ensure that the CFL number of the Coriolis force term and the convection term ≤ 1. The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge of those skilled in the art without departing from the gist of the present invention.

Claims

1. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling, characterized in that, It includes the following specific implementation steps: S1. Obtain satellite remote sensing data, real-time synchronized meteorological data, and fuse IMU data and GPS data through extended Kalman filtering; S2. Use Kalman filtering to smooth the temporal differences of multi-source data, complement satellite data with a delay exceeding 5 seconds through cubic spline interpolation, and dynamically adjust the process noise covariance matrix; S3. Construct an aerosol-humidity coupled illumination model to dynamically calculate solar illumination, including modeling the light source direction based on the solar declination angle and hour angle, adjusting the light intensity attenuation coefficient according to the environmental humidity, constructing a wind field dynamics simulation to generate a wind speed field, driving the scene elements to respond to wind field changes, using RTX hardware acceleration for ray tracing and U-Net denoising model for ray tracing and denoising, and performing dynamic texture management, and simulating cloud optical effects; S4. Access the virtual environment through a WebXR device, real-time synchronize the user's head pose data with a delay less than 20 ms, and combine gesture recognition and force feedback mechanisms, extract gesture features based on the Leap Motion sensor, and generate collision feedback through the Cannon.js physics engine.

2. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to claim 1, characterized in that, The preprocessing of satellite remote sensing data includes: collecting point cloud data of the target area through a Velodyne VLS-128 lidar; using the iterative closest point algorithm to minimize the spatial deviation between the satellite image and the point cloud, outputting the rotation matrix R and translation vector t, and iteratively optimizing until convergence to output the calibrated geographical coordinates.

3. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to claim 1, characterized in that, The specific steps for fusing IMU data and GPS data include: S31. Predict the system position and velocity through the state equation: where X k+1 represents the position at time k + 1; x k represents the position at time k; represents the velocity of the system at time k; represents the acceleration at time k; Δt represents the discrete time step; represents the velocity at time k + 1; S32. Use GPS data as the observation value, and set the covariance matrix as R = diag(1.0, 1.0, 1.0); Among them, diag() represents a diagonal matrix.

4. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to claim 1, characterized in that, The parameter settings of Kalman filtering are: For IMU data, the process noise covariance matrix Q = diag(0.5, 0.5, 0.5, 0.1, 0.1, 0.1); For meteorological data, the process noise Q = 0.1; When the noise is non-Gaussian, switch to the variational parameter variational Bayesian Kalman filtering algorithm VBKF and dynamically adjust the variational parameters; Among them, diag() represents a diagonal matrix.

5. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to claim 1, characterized in that The wind field dynamics simulation includes: Perform three-dimensional fluid simulation using the LBM and D3Q27 lattice model; Implement the collision and flow calculation of the distribution function through WebGL Compute Shader; Add the temperature gradient external force term as a source term to the LBM evolution equation to drive the dynamic response of the scene elements.

6. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to claim 1, characterized in that Ray tracing and denoising include: performing 64 ray samples per pixel, with a recursive depth of 3 times, and using a pre-trained model with a U-Net structure to denoise the noisy frames.

7. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to claim 1, characterized in that The aerosol-humidity coupled illumination model is: S71. Dynamic calculation of solar illumination: Modeling of the solar position: Calculation of the declination angle and hour angle: In the formula, σ represents the solar declination angle; n represents the day number of the year; H represents the solar hour angle; h UTC represents Coordinated Universal Time; λ represents the local longitude; E time represents the time difference, the difference between the true solar time and the mean solar time, which corrects the uneven apparent motion caused by the Earth's orbital revolution and the tilt of the axis of rotation; S72. Light intensity attenuation: Based on the humidity absorption coefficient a = 0.02·(1 + humidity / 100), dynamically adjust the intensity of the parallel light: Wherein, I represents the attenuated light intensity; I0 represents the initial light intensity; H' represents the atmospheric equivalent thickness; humidity represents the environmental humidity; e represents the natural constant; V represents the environmental visibility; Q txt represents the aerosol extinction coefficient; r represents the aerosol particle radius; λ is the light wavelength; m is the aerosol complex refractive index; S73. When the visibility V in the meteorological data is less than 10 km, activate the Mie scattering calculation; In the fragment shader, according to V and Q ext Dynamically adjust the light attenuation intensity to simulate the scattering effect of haze / dust; S74. Three.js light source setting: According to the solar azimuth and altitude angles, update the direction and color of the parallel light in real time.

8. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to claim 1, characterized in that The wind field dynamics simulation also includes: Introduce the Coriolis force term into the Navier-Stokes equations where Ω is the angular velocity vector of the Earth's rotation; Dynamically adjust the components of Ω according to the latitude φ where the user is located, and the direction is determined by the right-hand rule; Explicitly discretize the Coriolis force term by the finite difference method.

9. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to claim 1, characterized in that The force feedback mechanism includes: Driving the device to vibrate at a frequency of 200 Hz through the WebHaptics API, and the vibration duration is proportional to the magnitude of the virtual collision force.

10. A centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling, which is used to execute the centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling according to any one of claims 1 to 9, and is characterized in that, Including: The data acquisition layer, which obtains satellite remote sensing data and real-time data from meteorological stations; The data fusion layer, which uses Kalman filtering to smooth time series data and uses variational Bayesian Kalman filtering (VBKF) to process non-Gaussian noise; The rendering engine layer, which dynamically calculates the sunlight, adjusts the light intensity attenuation coefficient according to the environmental humidity, constructs the wind field dynamics simulation, generates the wind speed field, and drives the scene elements to respond to the wind field changes, performs ray tracing and noise reduction, performs dynamic texture management, and simulates the optical effects of clouds; The VR interaction layer, which accelerates the fluid simulation and realizes the dynamic change of the wind field.

Images