A virtual laboratory simulation system and method based on VR technology

Abstract

The application relates to the field of virtual reality simulation, and discloses a virtual laboratory simulation system and method based on VR technology, which comprises the following steps: obtaining user interaction data through a sensor for preprocessing, transmitting the user interaction data in real time through a network communication protocol, carrying out data fusion and synchronous processing to update scene data, and transmitting the updated scene data to the user; calculating a scene synchronization value through network delay and device performance difference, carrying out synchronization verification on the scene data in space-time three dimensions, adopting a scene state synchronization algorithm based on a distributed hash table to monitor the change of the scene state and dynamically adjust; detecting and compensating the operation delay of the user, establishing an operation delay model, predicting the operation action of the user, and executing in advance in the virtual scene; generating a friction haptic feedback signal of interaction in real time according to the interaction data of objects in the virtual scene, and controlling the haptic feedback in combination with a pressure sensor and a vibration motor hardware device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of virtual reality simulation, and in particular to a virtual laboratory simulation system and method based on VR technology. BACKGROUND

[0002] Virtual reality (VR) technology brings progress to the fields of education, scientific research and industry through immersive interaction and three-dimensional visualization. In the field of education, a VR virtual laboratory breaks through the time and space limitations of traditional experiments, allowing students to conduct high-risk or high-cost experiments in a safe and low-cost environment. For example, students can operate chemical hazardous substances or simulate nuclear reactions. A VR laboratory for architecture colleges helps students intuitively understand architectural structures and light and shadow effects through high-precision modeling and spatial positioning technology. A flight simulation laboratory improves the efficiency and safety of pilot training by combining haptic feedback and brain wave monitoring. The physical risks in real experiments are avoided, and multi-degree-of-freedom operations such as disassembling engines and adjusting experimental parameters are performed in a virtual environment. Multiple users can participate in synchronous operations in the same virtual space to collaboratively design or analyze data. Cloud rendering technology enables real-time access across devices and across regions.

[0003] In the prior art, traditional solutions often use a centralized server to synchronize scene states. Based on multi-user synchronous participation, the server has high load and a single point of failure risk. When the number of users increases, synchronization delays tend to increase rapidly. In addition, the synchronization accuracy is inconsistent due to differences in device performance. Scene synchronization is only based on position coordinates, ignoring object poses and timing logic such as operation execution sequences, which leads to visual synchronization but action misplacement. For example, when two people simultaneously push and pull the same object, the local display action sequence contradicts the actual physical logic. Simple time stamps or version numbers are used as priorities without considering experimental process compliance or device reliability, which leads to the adoption of incorrect operations. Linear prediction or basic Kalman filtering is commonly used to handle uniform or uniform acceleration motion, resulting in large prediction errors for complex gestures or non-linear trajectories, which leads to virtual action lag. The Coulomb friction formula with a fixed friction coefficient is used, ignoring the dynamic characteristics of the friction coefficient changing with sliding speed and not considering the influence of contact area and surface roughness on friction, which leads to distortion of the virtual object's tactile sensation.

[0004] Therefore, there is a need to provide a virtual laboratory simulation system and method based on VR technology. SUMMARY

[0005] The present application aims to provide a virtual laboratory simulation system and method based on VR technology. To solve the above-mentioned problems in the prior art, the present application achieves the following technical solutions:

[0006] In a first aspect, the present application provides a virtual laboratory simulation system based on VR technology, which specifically includes the following modules:

[0007] Multi-person interaction module: pre-process the user interaction data obtained through sensors, transmit the user's interaction data in real time through network communication protocol, perform data fusion and synchronization processing to update scene data, and transmit the updated scene data to the user;

[0008] Scene adjustment module: calculate scene synchronization value through network delay and device performance difference, perform synchronization verification based on spatio-temporal three-dimensional scene data fusion, adopt scene state synchronization algorithm based on distributed hash table, monitor changes in scene state and perform dynamic adjustment;

[0009] Delay compensation module: detect and compensate the user's operation delay, predict the user's operation action by establishing an operation delay model, and perform the action in advance in the virtual scene;

[0010] Dynamic feedback module: generate interactive friction haptic feedback signals in real time according to the interaction data of objects in the virtual scene, and control haptic feedback in combination with pressure sensors and vibration motor hardware devices.

[0011] Further, the method for data fusion and synchronization processing to update scene data is:

[0012] Use data fusion algorithm to integrate the same type of user interaction data from different users; obtain multi-user operation data of the same object, including coordinate position and attitude quaternion; for obtaining the coordinate position of the object, use weighted least squares method to fuse the position coordinates, assign different weights according to the device accuracy, and calculate the comprehensive position of the object in the virtual scene; for obtaining the attitude quaternion, use quaternion fusion method to fuse the attitude information of the same object from multiple users, and update the object attitude; the fused data forms complete scene state data;

[0013] Define scene operation event structure, and map the event to DHT node through synchronization hash algorithm;

[0014] Further, the method for calculating scene synchronization value is:

[0015] Obtain local scene coordinates and server state coordinates , calculate the Euclidean distance between the local scene and the server state, and obtain the Euclidean distance between the local scene and the server state Normalize and de-dimension to obtain position error value ;

[0016] Based on the attitude quaternion of the local scene and the server state, calculate the quaternion spherical distance by the formula: Normalize and de-dimension to obtain attitude error value ;​

[0017] Subtracting the local scene time from the server time and taking the absolute value to obtain a clock offset, and performing ratio calculation on the obtained clock offset and a preset standard time amount to obtain a time error value ;

[0018] Comprehensively processing the obtained position error value, attitude error value and time error value, and obtaining a comprehensive error value through weighted fusion ;

[0019] Based on the obtained comprehensive error, calculating to obtain a scene synchronization value ;

[0020] Further, the method for checking synchronization is:

[0021] Performing a global check once in a check period, and if the scene synchronization value is less than a synchronization threshold value, triggering an adjustment mechanism;

[0022] Completing attitude adjustment through three rendering frames, and the adjustment amount of each frame ;

[0023] According to the network delay and the device performance score, dynamically adjusting the synchronization threshold value through the formula:

[0024] ;

[0025] Calculating the dynamically adjusted synchronization threshold value , wherein, is a one-way delay, is a preset basic synchronization threshold value, , is a preset adjustable coefficient, dominates the negative adjustment of the one-way delay, dominates the positive compensation of the device performance;

[0026] Further, the method for dynamically adjusting is:

[0027] Obtaining a user ID and a device fingerprint and hashing them into a 160-bit identifier, and mapping them to a ring space;

[0028] Hashing the scene state data according to the object ID, and determining the responsible node for each scene state data block by its hash value;

[0029] When the client detects a scene change, generating a change log;

[0030] Adjusting based on the one-way delay: if > 100 ms, reducing the rendering frame rate, and predicting the next position of the object based on historical data , render ahead:

[0031] ;

[0032] where, is the space-time vector at time t, is the velocity at time t, is the acceleration at time t, t is the time index, is the predicted latency;

[0033] Adjustment based on device performance: according to Dynamic adjustment of detail level;

[0034] Further, the method for detecting the delay is:

[0035] The client adds a timestamp T1 when sending operation data, the server records it as T2 when receiving, returns a timestamp T3 after processing, and the client records it as T4 when receiving;

[0036] Real-time calculation of each segment delay , data processing delay ;

[0037] Maintain a sliding window of the last K delay samples, calculate the delay mean and delay standard deviation ;

[0038] Based on the obtained delay standard deviation, identify the delay mode, if , it is identified as stable delay; if , it is identified as fluctuating delay;

[0039] Further, the method for predicting the user's operation action is:

[0040] Differential processing of position coordinates / attitude quaternions to generate velocity where, is the space-time vector at time t, is the space-time vector at time t-1, is the sampling interval;

[0041] Use normalization technology to scale numerical features to the [-1, 1] interval;

[0042] For uniform / constant acceleration motion scenarios, the prediction is made by the formula:

[0043] ;

[0044] Calculate the predicted space-time vector at t+1 ;

[0045] where, to predict the time interval, the acceleration at time t;

[0046] For complex action prediction, a two-layer LSTM neural network and a fully connected layer are used to input historical data with a sequence length of N, and the root mean square error is used as the loss function, and the Adam optimizer is used for optimization.

[0047] Further, the method for pre-execution is:

[0048] Collect M groups of real operation trajectories, judge the action complexity according to the calculated action complexity index, that is, the acceleration variance, and dynamically select the prediction model;

[0049] According to the total delay prediction value , wherein, is the data processing delay, is the rendering output delay, and the pre-execution time advance is determined ;

[0050] Based on the pre-execution time advance, the predicted trajectory in the future ΔT time period is analyzed:

[0051] ;

[0052] , wherein, is the number of prediction time intervals in the ΔT time period, is an integer;

[0053] Further, the method for controlling haptic feedback is:

[0054] According to the interaction data of the objects in the virtual scene, the friction haptic feedback signal of the interaction is generated in real time;

[0055] The haptic feedback signal is divided into high-frequency vibration signal and low-frequency pressure signal;

[0056] The high-frequency vibration signal intensity is proportional to the friction force change rate, and the low-frequency pressure signal intensity is proportional to the friction force size;

[0057] The contact area and relative speed of object interaction are obtained in real time through the physical engine in the virtual scene;

[0058] According to the contact area, the signal duration is adjusted, and the phase of the vibration signal is modulated in combination with the relative speed direction;

[0059] A flexible pressure sensor array is evenly distributed on the palm and finger parts of the data glove, and the sensor output voltage has a linear relationship with the pressure.

[0060] According to the calculated high-frequency vibration signal intensity and low-frequency pressure signal intensity, a corresponding PWM control signal is generated;

[0061] The micro vibration motor is an eccentric rotating mass (ERM) motor and a linear resonant actuator; the eccentric rotating mass motor generates centrifugal force by rotating eccentric blocks to achieve vibration.

[0062] In a second aspect, the embodiment of the present application provides a virtual laboratory simulation method based on VR technology, comprising the following steps:

[0063] Step one: pre-process the user interaction data obtained by the sensor, transmit the user interaction data in real time through the network communication protocol, and update the scene data through data fusion and synchronization processing, and transmit the updated scene data to the user;

[0064] Step two: calculate the scene synchronization value based on network delay and device performance difference, perform synchronization verification based on the spatiotemporal three-dimensional fusion of scene data, adopt a scene state synchronization algorithm based on a distributed hash table, monitor the changes of scene state and perform dynamic adjustment;

[0065] Step three: detect and compensate the operation delay of the user, predict the operation action of the user by establishing an operation delay model, and perform the operation in advance in the virtual scene;

[0066] Step four: generate the friction haptic feedback signal of interaction in real time according to the interaction data of the object in the virtual scene, and control the haptic feedback in combination with the pressure sensor and the vibration motor hardware device.

[0067] The present application has the following advantages:

[0068] 1. Multi-dimensional interaction data fusion improves immersion and accuracy, real-time network transmission and distributed collaboration ensure smoothness of multi-person interaction, dynamic scene synchronization and delay compensation adapt to complex network and device environment, and delay is decomposed into four stages of collection, network, processing and rendering, model prediction fluctuation delay, LSTM neural network processing complex motion trajectory, and motion prediction formula are combined to realize the pre-execution of user operation, effectively reduce the perception delay physical engine driven haptic feedback, and enhance the virtual interaction reality;

[0069] 2. Introduce the DHT node architecture, hash the scene state data according to the object ID, map it to the ring space node through the synchronization hash algorithm, realize the decentralized load balancing and data routing, compared with the traditional center server architecture, improve the system scalability and fault tolerance, suitable for large-scale multi-user virtual laboratory scene; Construct a three-dimensional error model, quantify the scene deviation through the Euclidean distance and the quaternion spherical distance, and dynamically adjust the synchronization threshold based on the network delay and device performance, realize the adaptability of the scene synchronization strategy, break through the limitations of the traditional fixed threshold synchronization, and classify processing according to stable delay and fluctuating delay: stable delay adopts mean prediction, fluctuating delay introduces time series model, combines historical delay data to predict the next period delay, and complex action learns trajectory features through two-layer LSTM neural network, which is more robust than single prediction model, and effectively solves the dizziness caused by delay in the operation delay interaction of VR users. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0071] Figure 1 It is a structure schematic diagram of a virtual laboratory simulation system based on VR technology provided by embodiment 1 of the present application.

[0072] Figure 2 It is a step flow chart of a virtual laboratory simulation method based on VR technology provided by embodiment 2 of the present application. DETAILED DESCRIPTION

[0073] In order to make the person in the art better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely in the following, obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0074] Embodiment 1:

[0075] As shown in Figure 1 , the virtual laboratory simulation system based on VR technology provided by the embodiment of the present application specifically includes the following modules:

[0076] Multi-person interaction module: the user interaction data obtained by the sensor is preprocessed, the user's interaction data is transmitted in real time through the network communication protocol, and the scene data is updated through data fusion and synchronization processing, and the updated scene data is transmitted to the user;

[0077] It should be noted that the interaction data includes but is not limited to: coordinate position, attitude quaternion, finger joint angle, palm pressure and voice data;

[0078] In specific embodiments, the method for obtaining user interaction data is:

[0079] The head coordinate position of the user based on the coordinate system of the spatial positioning base station is obtained by the camera, gyroscope and accelerometer sensor built in the VR helmet And the head attitude quaternion The hand coordinate position of the user is obtained by the sensor on the VR handle or data glove , combined with the built-in IMU hand attitude quaternion , the finger joint angle and palm pressure p are collected by the flexible sensor The user's voice data is obtained by the microphone for voice communication and instruction input;

[0080] It should be noted that the attitude quaternion includes: the rotation angle cosine value of the user about the unit vector, the pitch angle sine value, the yaw angle sine value and the rotation angle sine value;

[0081] The collected raw data is preprocessed, the position and attitude data of the head and hand are smoothed by using Kalman filtering algorithm, the noise interference is removed, and the data accuracy and stability are improved; The voice data is denoised and speech recognized, and the voice data is converted into a digital signal recognizable by a computer, laying a foundation for subsequent transmission and processing;

[0082] The joint angle of the handle / glove is normalized to the interval [0, 1], and the pressure value is converted to the relative pressure of the reference standard atmospheric pressure;

[0083] The hand attitude quaternion is unitized to ensure ;

[0084] UDP protocol is used for data transmission, which has the characteristics of strong real-time, which meets the requirements of virtual laboratory for data transmission speed; The data is compressed and encrypted to ensure transmission reliability, reduce data transmission amount and protect data security;

[0085] The data is transmitted to the server or local computer through the wireless network, before transmission, the transmission parameters are configured according to the network environment, such as setting the transmission port, IP address, to ensure that the data is accurately sent to the target address;

[0086] A dedicated data transmission channel is established between the user terminal and the server, each user corresponds to an independent channel identifier, ensuring the uniqueness and accuracy of data transmission; During the channel establishment process, handshake verification is performed to confirm that both devices are in normal state and have transmission conditions, preventing data transmission interruption or errors;

[0087] The server receives data from each user terminal through the data receiving module, parses it according to the preset data format, separates the different types of interactive data, and stores it in the corresponding temporary data storage area for subsequent processing;

[0088] Using data fusion algorithm, the same type of user interaction data from different users is integrated; Obtain multi-user operation data of the same object, operation data includes: coordinate position and attitude quaternion, for obtaining the coordinate position of the object, weighted least squares method is used to fuse position coordinates, different weights are given according to device accuracy, and the comprehensive position of the object in the virtual scene is calculated; For the attitude quaternion, the attitude information of the same object from multiple users is fused through the quaternion fusion method to update the object attitude; The fused data forms complete scene state data;

[0089] Define scene operation event structure, map events to DHT nodes through synchronization hash algorithm, and ensure load balancing;

[0090] It should be noted that DHT node is a decentralized distributed system architecture used to efficiently manage resources and data routing of nodes in the network without a central node;

[0091] Scene adjustment module: calculate scene synchronization value through network delay and device performance difference, perform synchronization verification based on scene data fusion space-time three-dimensional, use scene state synchronization algorithm based on distributed hash table, monitor scene state changes and perform dynamic adjustment;

[0092] In specific embodiments, the method for calculating scene synchronization value is:

[0093] The client and the server send and receive heartbeat packets with timestamps every 50ms to calculate the round-trip time RTT;

[0094] Specifically, the client records the sending time , the server records the receiving time and the packet return time , and the client records when receiving the packet, then the one-way delay is: ;

[0095] Collect the sliding window of the last 20 round-trip time RTT samples, calculate the median as the current network delay, and eliminate the impact of burst jitter;

[0096] CPU single / multi-threaded floating point operation capability when running performance test program for the first time GPU triangle generation rate and pixel fill rate Memory bandwidth and latency ;

[0097] Device parameters CPU single / multi-threaded floating point operation capability GPU triangle generation rate and pixel fill rate Memory bandwidth and latency Synthetic performance score mapped to [0, 1] The preset proportion coefficients are 0.4, 0.4, and 0.2.

[0098] Collect object position Quaternion of attitude Timestamp T, construct space-time vector ;

[0099] Synchronize multi-sensor data through hardware timestamp to ensure space-time synchronization.

[0100] Get local scene coordinates Server state coordinates Calculate the Euclidean distance between the local scene and the server state through the formula:

[0101] ;

[0102] Calculate the Euclidean distance between the local scene and the server state Normalize and de-dimension to get the position error value ;

[0103] Based on the quaternion of attitude of the local scene and the server state, calculate the spherical distance of the quaternion through the formula:

[0104] ;

[0105] Calculate the spherical distance of the quaternion Normalize and de-dimension to get the attitude error value ;

[0106] Subtract the local scene time from the server time and take the absolute value to get the clock offset. Calculate the ratio of the obtained clock offset to the preset standard time to de-dimension and get the time error value ;

[0107] Synthetically process the position error value, attitude error value, and time error value, and get the comprehensive error value by weighted fusion , the weights can be dynamically adjusted according to the scene, and when fine operation, the weights are respectively preset as: 0.6, 0.3, 0.1;

[0108] Based on the obtained comprehensive error, the scene synchronization value is calculated ;

[0109] A global check is performed once in a check period, and if the scene synchronization value is less than the synchronization threshold, the adjustment mechanism is triggered;

[0110] For example, for the error dominated by the attitude error, the interpolation attitude of the local scene attitude and the target attitude is calculated :

[0111] ;

[0112] Wherein, is the angle between the attitudes, , is the interpolation progress, ;

[0113] The attitude adjustment is completed through three rendering frames, and the adjustment amount of each frame is , to avoid visual jump;

[0114] The synchronization threshold is dynamically adjusted according to the network delay and the device performance score, through the formula:

[0115] ;

[0116] The dynamically adjusted synchronization threshold is calculated , wherein, is the preset basic synchronization threshold, , is the preset adjustable coefficient, preset as 0.08, 0.3, negative adjustment dominated by one-way delay, positive compensation dominated by device performance;

[0117] Get the user ID and device fingerprint and hash it into a 160-bit identifier, which is mapped to a ring space;

[0118] Hash the scene state data according to the object ID, and each scene state data block is determined by its hash value. The responsible node is determined by its hash value;

[0119] When the client detects the scene change, a change log is generated ;

[0120] Wherein, is the object ID hash, is the space-time vector change,​ to change the timestamp;

[0121] Adjustment based on one-way latency: if > 100 ms, reduce the rendering frame rate, predict the next position of the object based on historical data , render in advance to reduce perceived latency:

[0122] ;

[0123] wherein, is the space-time vector at time t, is the velocity at time t, is the acceleration at time t, t is the time index, is the estimated latency;

[0124] Adjustment based on device performance: according to dynamically adjust the level of detail, for example, if is greater than 80, adjust to high detail; if is greater than 50 and less than or equal to 80, adjust to medium detail; if is less than or equal to 50, adjust to low detail;

[0125] Latency compensation module: detect and compensate the operation latency of the user, predict the operation action of the user by establishing an operation latency model, and perform in advance in the virtual scene;

[0126] In specific embodiments, the method for detecting and compensating the operation latency of the user is:

[0127] Obtain the time from sensor data acquisition to data packaging to obtain the input acquisition latency ;

[0128] Calculate the network transmission latency by calculating 1 / 2 of the round-trip time RTT from the client to the server through the two-way heartbeat packet ;

[0129] Obtain the data processing latency by obtaining the time for server-side data fusion and physical simulation ;

[0130] Obtain the rendering output latency by obtaining the hardware rendering latency from scene data to VR headset display ;

[0131] The client adds a timestamp T1 when sending operation data, the server records it as T2 when receiving, and returns a timestamp T3 after processing, which is recorded as T4 by the client when receiving;

[0132] Real-time calculation of each segment latency , data processing latency ;

[0133] Maintain a sliding window of the last K delay samples, calculate the delay mean and delay standard deviation ;

[0134] Identify the delay pattern based on the obtained delay standard deviation, if , identify as stable delay; if , identify as fluctuant delay;

[0135] For stable delay, use the mean as the predicted value;

[0136] For fluctuant delay, use the ARIMA time series model to predict the next period delay, through the formula:

[0137] ;

[0138] Calculate the predicted delay at t+1 time;

[0139] Wherein, is the network transmission delay at t time, is the network transmission delay at t-1 time, is the network transmission delay at t-2 time, t is the sampling time point, , , is the relevant proportion coefficient obtained by least square training, is white noise;

[0140] Differential processing of position coordinates / attitude quaternions to generate velocity , wherein, is the space-time vector at t time, is the space-time vector at t-1 time, is the sampling interval;

[0141] Use normalization technology to scale numerical features to the interval [-1, 1];

[0142] For uniform / constant acceleration motion scene prediction, through the formula:

[0143] ;

[0144] Calculate the predicted space-time vector at t+1 time ;

[0145] Wherein, is the predicted time interval, is the acceleration at t time;

[0146] For the prediction of complex actions, a two-layer LSTM neural network and a fully connected layer are used to input historical data with a sequence length of N, and the root mean square error is used as the loss function, which is optimized by the Adam optimizer.

[0147] M groups of real operation trajectories are collected, and the action complexity is determined according to the calculated action complexity index, i.e., the acceleration variance, and the prediction model is dynamically selected;

[0148] According to the total delay prediction value , the pre-execution time advance is determined;

[0149] Based on the pre-execution time advance, the predicted trajectory in the future ΔT time period is analyzed:

[0150] ;

[0151] Wherein, is the number of prediction time intervals in the ΔT time period, is an integer;

[0152] Dynamic feedback module: according to the interaction data of objects in the virtual scene, real-time generation of interactive friction haptic feedback signal, combined with pressure sensor and vibration motor hardware device control haptic feedback;

[0153] According to the interaction data of objects in the virtual scene, real-time generation of interactive friction haptic feedback signal, through the physical engine in the virtual scene to obtain the object interaction information in real time, including: the contact point coordinates , contact area A, relative speed , relative acceleration , record the object material friction Surface roughness ;

[0154] Use the pressure sensor on the data glove to collect hand area pressure data, divide the palm into 9 pressure sensing areas, each area corresponds to a pressure value , …, ;

[0155] Get the hand pose quaternion and hand position coordinates through the inertial measurement unit IMU, the sampling frequency is preset to 200Hz to ensure real-time data;

[0156] The 3σ principle is used to remove outliers from the pressure sensor data, if the pressure value of the pressure sensing area satisfies , wherein, is the average pressure of the area, is the standard deviation of the pressure in the area, For the region index, the pressure sensing region pressure value is regarded as an abnormal value, and a weighted average value of the pressure of adjacent regions is used to replace it;

[0157] The pressure data is normalized to the interval [0, 1], and the relative speed and acceleration are unit converted and standardized;

[0158] A friction calculation model is constructed, and the friction force at the contact point is calculated according to the Coulomb friction law ; The dynamic friction coefficient related to the speed is introduced, and the dynamic friction coefficient is calculated by the formula ;

[0159] wherein, is the sliding friction coefficient, is the dynamic friction coefficient, is the relative speed, is the speed threshold value;

[0160] The tactile feedback signal is divided into a high-frequency vibration signal and a low-frequency pressure signal;

[0161] The high-frequency vibration signal intensity is proportional to the friction force change rate, and the low-frequency pressure signal intensity is proportional to the friction force size;

[0162] According to the contact area A, the signal duration is adjusted, the larger the contact area is, the longer the signal duration T is; the phase of the vibration signal is modulated in combination with the relative speed direction, so that the tactile feedback has a direction sense;

[0163] A flexible pressure sensor array is evenly distributed on the palm and finger parts of the data glove, the sensor output voltage has a linear relationship with the pressure, and the analog signal is converted into a digital signal by A / D conversion and input into the control system;

[0164] According to the calculated high-frequency vibration signal intensity and low-frequency pressure signal intensity, corresponding PWM control signals are generated;

[0165] Through a miniature vibration motor, an eccentric rotating mass ERM motor and a linear resonant actuator are used; the eccentric rotating mass motor generates centrifugal force by rotating the eccentric block to realize vibration;

[0166] When multiple vibration motors and pressure sensors work at the same time, time division multiplexing technology is used for control to avoid signal interference.

[0167] Embodiment 2:

[0168] As Figure 2 shown, the virtual laboratory simulation method based on the VR technology provided by the embodiment of the application comprises the following steps:

[0169] Step one: pre-process the user interaction data obtained by the sensor, transmit the user interaction data in real time through the network communication protocol, and update the scene data through data fusion and synchronization processing, and transmit the updated scene data to the user;

[0170] Step two: calculate the scene synchronization value through network delay and device performance difference, perform synchronization verification based on the spatiotemporal three-dimensional fusion of scene data, use a scene state synchronization algorithm based on a distributed hash table to monitor the change of the scene state and dynamically adjust it;

[0171] Step three: detect and compensate the operation delay of the user, predict the operation action of the user by establishing an operation delay model, and perform the operation in advance in the virtual scene;

[0172] Step four: generate the interactive friction haptic feedback signal in real time according to the interaction data of the object in the virtual scene, and control the haptic feedback in combination with the pressure sensor and the vibration motor hardware device.

[0173] The above describes one embodiment of the present application in detail, but the content described is only a preferred embodiment of the present application and cannot be considered as limiting the scope of the present application; the above formulas are all dimensionless values calculated, the formulas are obtained by software simulation of a large amount of collected data to obtain a formula of the most recent real situation, the preset parameters in the formula are set by the person skilled in the art according to the actual situation and historical experience, and can be adjusted according to the actual situation; the above described is only a preferred embodiment of the present application and cannot be considered as limiting the present application, and any equivalent changes and improvements made according to the scope of the present application should still belong to the patent coverage range of the present application.

Claims

1. A virtual laboratory simulation system based on VR technology, characterized in that, Includes the following modules: Multi-user interaction module: It acquires user interaction data through sensors, preprocesses it, transmits the user interaction data in real time through network communication protocols, performs data fusion and synchronous processing to update scene data, and transmits the updated scene data to the user. Scene adjustment module: Analyzes network latency and device performance differences and calculates scene synchronization value. It performs synchronization verification based on spatiotemporal 3D fusion of scene data and adopts a scene state synchronization algorithm based on distributed hash table to monitor changes in scene state and make dynamic adjustments. The specific method of using the scene state synchronization algorithm based on a distributed hash table is as follows: Obtain the user ID and device fingerprint, hash them into a 160-bit identifier, and map them to a ring space; The scene state data is sharded by object ID hash, and the responsible node for each scene state data block is determined by its hash value. When the client detects a change in the scene, it generates a change log. Adjustment based on unidirectional delay: If If the time frame exceeds 100ms, reduce the rendering frame rate and predict the object's next position based on historical data. Pre-rendering: ； in, Let be the spacetime vector at time t. Let be the velocity at time t. Let be the acceleration at time t, where t is the time index. To estimate the delay; Adjustments based on device performance: Dynamically adjust the level of detail based on performance scores; Delay compensation module: Detects and compensates for user operation delays. By establishing an operation delay model, it predicts user actions and executes them in advance in the virtual scene. Dynamic feedback module: Based on the interaction data of objects in the virtual scene, it generates interactive friction and tactile feedback signals in real time, and controls the tactile feedback in combination with pressure sensors and vibration motor hardware devices.

2. The virtual laboratory simulation system based on VR technology according to claim 1, characterized in that, The method for data fusion and synchronous processing of updated scenario data is as follows: By using data fusion algorithms, similar user interaction data from different users can be integrated; The system acquires multi-user operation data for the same object, including coordinate position and pose quaternions. For acquiring the object's coordinate position, a weighted least squares method is used to fuse the position coordinates, assigning different weights according to device accuracy to calculate the object's comprehensive position in the virtual scene. For acquiring the pose quaternions, a quaternion fusion method is used to fuse the pose information of multiple users on the same object and update the object's pose. The fused data forms complete scene state data. Define a scene operation event structure and map the events to DHT nodes using a synchronous hash algorithm.

3. The virtual laboratory simulation system based on VR technology according to claim 1, characterized in that, The method for calculating the scene synchronization value is as follows: Get local scene coordinates server status coordinates Calculate the Euclidean distance between the local scene and the server state. Normalization and dimensionless determination yield the position error value ; The pose quaternion based on the local scenario and server status is obtained through the formula: The quaternion spherical distance was calculated. Normalization and dimensionless determination yield the attitude error value ; The local scene time is obtained and the server time is subtracted, and the absolute value is taken to obtain the clock offset. The ratio of the obtained clock offset to a preset standard time is calculated and then dedivided to obtain the time error value. ; The obtained position error, attitude error, and time error values ​​are processed together and weighted to obtain the comprehensive error value. ; Based on the obtained comprehensive error, calculate Get scene synchronization value .

4. The virtual laboratory simulation system based on VR technology according to claim 1, characterized in that, The method for performing synchronization verification is as follows: A global check is performed once during the check period. If the scene synchronization value is less than the synchronization threshold, an adjustment mechanism is triggered. The pose adjustment is completed in 3 rendering frames, with the adjustment amount per frame. ; The synchronization threshold is dynamically adjusted based on network latency and device performance score, using the formula: ； The dynamically adjusted synchronization threshold is calculated. ,in, It is a one-way delay. To preset the basic synchronization threshold, , For preset adjustable coefficients, The negative adjustment that dominates unidirectional delay. Positive compensation for the performance of the main equipment.

5. A virtual laboratory simulation system based on VR technology according to claim 1, characterized in that, The method for detecting the delay is as follows: When the client sends operation data, it adds a timestamp T1. When the server receives the data, it records it as T2. After processing, it returns a timestamp T3, which the client records as T4 when it receives the data. Real-time calculation of delay for each segment Data processing delay ; Maintain a sliding window for the K most recent delayed samples and calculate the mean delay. and delay standard deviation ; Based on the obtained delay standard deviation, the delay pattern is identified. If, then it is identified as a stable delay; if If so, it is identified as fluctuation delay.

6. A virtual laboratory simulation system based on VR technology according to claim 1, characterized in that, The method for predicting user actions is as follows: Perform differential processing on the position coordinates / attitude quaternions to generate velocity. ,in, Let be the spacetime vector at time t. Let be the spacetime vector at time t-1. The sampling interval; The numerical features are scaled to the [-1,1] interval using normalization techniques; For prediction of uniform / uniformly accelerated motion scenarios, the formula is: ； The predicted spatiotemporal vector at time t+1 is calculated. ; in, For predicting time intervals, Let be the acceleration at time t; For predicting complex actions, a two-layer LSTM neural network and a fully connected layer are used. The input sequence is historical data of length N, and the root mean square error is used as the loss function. The result is optimized by the Adam optimizer.

7. A virtual laboratory simulation system based on VR technology according to claim 1, characterized in that, The method for early execution is as follows: Collect M groups of real operation trajectories, determine the action complexity based on the calculated action complexity index, i.e., acceleration variance, and dynamically select the prediction model; Based on total delay forecast ,in, Due to data processing delay, Determine the pre-execution time advance to delay rendering output. ; Based on the lead time for pre-execution, the predicted trajectory within the future time period ΔT is obtained through analysis: ； in, This represents the number of predicted time intervals within the time interval ΔT. It is an integer.

8. A virtual laboratory simulation system based on VR technology according to claim 1, characterized in that, The method for controlling tactile feedback is as follows: Based on the interaction data of objects in the virtual scene, generate real-time friction and tactile feedback signals for interaction; The tactile feedback signal is divided into high-frequency vibration signal and low-frequency pressure signal; The intensity of high-frequency vibration signals is directly proportional to the rate of change of frictional force, while the intensity of low-frequency pressure signals is directly proportional to the magnitude of frictional force. The physical engine in the virtual scene obtains the contact area and relative speed of object interactions in real time. The signal duration is adjusted according to the contact area, and the phase of the vibration signal is modulated in combination with the relative velocity direction; A flexible pressure sensor array is used, evenly distributed in the palm and fingers of the data glove, and the sensor output voltage has a linear relationship with the pressure. Based on the calculated high-frequency vibration signal intensity and low-frequency pressure signal intensity, a corresponding PWM control signal is generated. The micro-vibration motor is used as an eccentric rotating mass ERM motor and a linear resonant actuator; the eccentric rotating mass motor generates centrifugal force through the rotation of the eccentric block to achieve vibration.

9. A virtual laboratory simulation method based on VR technology, applied to a virtual laboratory simulation system based on VR technology according to any one of claims 1-8, characterized in that, Includes the following steps: Step 1: Acquire user interaction data through sensors for preprocessing, transmit the user interaction data in real time through network communication protocols, perform data fusion and synchronous processing to update scene data, and transmit the updated scene data to the user; Step 2: Calculate the scene synchronization value based on network latency and device performance differences, perform synchronization verification based on scene data fusion in spatiotemporal three dimensions, and adopt a scene state synchronization algorithm based on a distributed hash table to monitor changes in scene state and make dynamic adjustments. Step 3: Detect and compensate for user operation delays. By establishing an operation delay model, predict user actions and execute them in advance in the virtual scene. Step 4: Based on the interaction data of objects in the virtual scene, generate interactive friction tactile feedback signals in real time, and control the tactile feedback by combining pressure sensors and vibration motor hardware devices.

Images