A VR teaching experience enhancement system and method

Abstract

The present invention discloses a VR teaching experience enhancement system and method, which belongs to the field of virtual teaching technology. The method specifically includes: collecting position data and posture data of trainees in the VR environment, constructing a multi-user coordinated interaction model based on the position data and posture data of the trainees in the VR environment, identifying the interaction relationship and interaction events between the trainees, generating tactile feedback data corresponding to the interaction events based on the interaction events, performing calibration processing on the tactile feedback data, and synchronously sending the calibrated tactile feedback data to the trainee's VR device, the calibration processing including adjusting the tactile feedback data based on the spatial propagation difference amount and tactile feedback time offset value of the trainee; the present application enables multiple trainees to obtain consistent, real-time tactile response and spatial perception when collaboratively operating the same virtual object or scene, thereby improving the teaching experience in multi-person collaborative scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of virtual teaching technology, and specifically relates to a VR teaching experience enhancement system and method. Background Art

[0002] Existing VR teaching systems are mostly based on three-dimensional scene modeling and motion capture technology, combined with multimodal information such as voice and images to achieve intuitive presentation of teaching content and basic interactive experience.

[0003] In single-person teaching tasks, VR systems typically track the trainee's head and hand positions through spatial positioning devices and interact with virtual objects, which can effectively meet basic training needs. However, as teaching tasks shift towards collaborative and group learning, existing technologies have the following major shortcomings in teaching scenarios involving multiple people working together: spatial positioning data is difficult to integrate in real time, tactile feedback cannot respond synchronously, and device latency and spatial differences are not compensated.

[0004] Therefore, based on the shortcomings of existing technologies, a new method to enhance VR teaching experience is urgently needed. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention proposes a VR teaching experience enhancement system and method.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for enhancing VR teaching experience, comprising:

[0008] Collect the position and posture data of trainees in the VR environment;

[0009] Based on the position data and posture data of the trainees in the VR environment, a multi-user coordinated interaction model is constructed, wherein the multi-user coordinated interaction model is used to integrate the spatial positioning data and posture data of the trainees and identify the interaction relationships and interaction events between the trainees;

[0010] Based on the interaction event, generating tactile feedback data corresponding to the interaction event;

[0011] Calibration processing is performed on the tactile feedback data, and the calibrated tactile feedback data is synchronously sent to the VR device of the trainee, wherein the calibration processing includes adjusting the tactile feedback data based on the spatial propagation difference amount and the tactile feedback time offset value of the trainee.

[0012] Specifically, the multi-user coordinated interaction model is constructed based on the position data and posture data of the trainees in the VR environment, including:

[0013] Perform time stamp synchronization on the position and posture data of trainees in the VR environment;

[0014] Based on the processed position and posture data of the trainees in the VR environment, the three-dimensional coordinate mapping and posture calculation of the trainees' spatial position information are performed, and the spatial state vector of the trainees is constructed;

[0015] Based on the spatial state vector, the relative position relationship and interaction movement trajectory between the trainees are analyzed to determine whether there are conditions for an interaction event to occur between the trainees;

[0016] When it is detected that the interaction actions between two or more trainees meet the preset collaborative interaction rules, it is determined to be a type of interaction event, and the interaction event is classified and processed.

[0017] Specifically, the three-dimensional coordinate mapping and posture calculation of the spatial position information of the trainee are performed based on the processed position data and posture data of the trainee in the VR environment, and the spatial state vector of the trainee is constructed, including:

[0018] Converting the position data of the trainee in the VR environment into three-dimensional coordinate values ​​in the virtual environment, wherein the conversion is performed based on a preset coordinate mapping matrix or calibration parameters;

[0019] The three-dimensional coordinate values ​​are integrated with the posture data of the trainee in the VR environment to construct the spatial state vector of the trainee at the current frame time. The spatial state vector is a combination vector of the position vector and the posture vector;

[0020] The spatial state vector is normalized.

[0021] Specifically, the analyzing the relative position relationship and interaction trajectory between the trainees based on the spatial state vector to determine whether conditions for an interaction event to occur between the trainees exist includes:

[0022] Compare the spatial state vectors of any two or more trainees in the same time frame and calculate the spatial distance, relative orientation angle and posture similarity between the trainees;

[0023] According to a preset interaction determination threshold, determining whether the spatial distance is lower than a first distance threshold, whether the relative orientation angle is within a preset angle range, and whether the posture similarity exceeds an action matching threshold;

[0024] If the preset interaction determination threshold is met, the trend of the trainee's motion trajectory in the adjacent time window is analyzed. The motion trajectory is extracted based on the time series composed of continuous spatial state vectors.

[0025] The motion trajectory is matched with a predefined interaction event pattern library. If the matching result reaches a set confidence threshold, it is determined that an interaction event has occurred between the trainees.

[0026] Specifically, generating tactile feedback data corresponding to the interaction event based on the interaction event includes:

[0027] According to the interaction event, a tactile feedback template matching the interaction event type is selected from a preset tactile feedback template library, wherein the tactile feedback template includes vibration frequency, amplitude, duration, and feedback mode;

[0028] Calculate the trainee's tactile feedback position, direction, and tactile intensity parameters to generate personalized adjustment feedback data;

[0029] Based on the tactile feedback template and the personalized adjustment feedback data, a tactile feedback data packet for the trainees is generated, and the data packet includes: feedback type, target device number, control instruction format, and execution timing information.

[0030] Specifically, performing calibration processing on the tactile feedback data includes:

[0031] Acquiring tactile feedback response parameters of the trainee, wherein the response parameters include data reception delay, drive execution delay, and device triggering delay;

[0032] Calculating the spatial propagation difference between the trainees based on their spatial state vectors at the time of the interaction event, where the spatial propagation difference is the relative propagation time difference of the tactile feedback data in the virtual scene;

[0033] The tactile feedback response parameter of the trainee is integrated with the spatial propagation difference to calculate the tactile feedback time offset value of the trainee;

[0034] According to the time offset value, the execution timing information in the tactile feedback data packet of the trainee is adjusted so that the trainee can synchronously trigger the tactile feedback at the target time point.

[0035] Specifically, the calculation of the spatial propagation difference between the trainees based on the spatial state vectors of the trainees at the time of the interaction event includes:

[0036] Determine the trigger center point of the interaction event, which is calculated based on the spatial coordinates of all trainees and can be a geometric center of mass, an event contact point, or an interaction reference point defined by the system;

[0037] Calculate the distance between each trainee and the trigger center point of the interaction event and set it as the propagation path length;

[0038] Based on the propagation path length and the simulated propagation speed of the tactile signal in the VR scene, the relative propagation delay of the trainees is calculated;

[0039] Taking the minimum value of all relative propagation delays as a reference, the propagation difference of the trainee relative to the earliest responding trainee is calculated, which is the spatial propagation difference.

[0040] Specifically, the tactile feedback response parameter of the trainee is integrated with the spatial propagation difference to calculate the tactile feedback time offset value of the trainee, including:

[0041] Calculating a total feedback delay of the trainee, where the total feedback delay is obtained by weighted summation of communication delay, device execution delay, and spatial propagation delay;

[0042] The minimum value of all total feedback delays is selected as the reference time point of global tactile feedback;

[0043] The difference between the total feedback delay of the trainee and the reference time point of the global tactile feedback is used as the tactile feedback time offset value of the trainee.

[0044] A VR teaching experience enhancement system, used to implement the VR teaching experience enhancement method, comprising: a data acquisition module, an interaction recognition module, a tactile feedback generation module, and a calibration and synchronization module;

[0045] The data acquisition module is used to collect the position data and posture data of the trainees in the VR environment;

[0046] The interaction recognition module builds a multi-user coordinated interaction model based on the position data and posture data of the trainees in the VR environment;

[0047] The tactile feedback generating module generates tactile feedback data corresponding to the interaction event based on the interaction event;

[0048] The calibration and synchronization module is used to perform calibration processing on the tactile feedback data and synchronously send the calibrated tactile feedback data to the VR device of the trainee.

[0049] Specifically, the calibration synchronization module includes: a response parameter acquisition unit, a propagation time difference calculation unit and a calibration adjustment unit;

[0050] The response parameter acquisition unit is used to obtain the tactile feedback response parameters of the trainees;

[0051] The propagation time difference calculation unit calculates the spatial propagation difference between the trainees based on the spatial state vectors of the trainees at the time of the interaction event;

[0052] The calibration adjustment unit is used to fuse the tactile feedback response parameters of the trainee with the spatial propagation difference, calculate the tactile feedback time offset value of the trainee, and adjust the execution timing information in the tactile feedback data packet of the trainee according to the time offset value, so that the trainee can synchronously trigger the tactile feedback at the target time point.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention proposes a VR teaching experience enhancement system and method, which can achieve efficient fusion and precise synchronization of spatial positioning data and tactile feedback data of multiple trainees in a virtual teaching environment. By constructing a coordinated interaction model, a tactile feedback template library and a delay compensation mechanism, multiple trainees can obtain consistent, real-time tactile response and spatial perception when collaboratively operating the same virtual object or scene, thereby significantly improving the immersion, interaction consistency and teaching experience in multi-person collaborative scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart of a method for enhancing VR teaching experience provided by the present invention;

[0056] Figure 2 Schematic diagram of the VR teaching environment provided by the present invention;

[0057] Figure 3 A schematic diagram of tactile feedback calibration synchronization provided by the present invention;

[0058] Figure 4 This is an architecture diagram of a VR teaching experience enhancement system provided by the present invention. DETAILED DESCRIPTION

[0059] The present application is described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but are not intended to limit the present application in any form. It should be noted that those skilled in the art may make several variations and improvements without departing from the scope of the present application. These all fall within the scope of protection of the present application.

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other and are all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flow chart. In addition, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish between the same items or similar items with basically the same functions and effects.

[0062] Unless otherwise defined, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which this application belongs. The terms used in this specification and in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the relevant listed items.

[0063] Example 1

[0064] See also Figure 1-Figure 3 An embodiment of the present invention provides a method for enhancing VR teaching experience. The method is run in a VR system equipped with a head display device, a position tracking device, and a tactile feedback device (such as a tactile glove or a vibration feedback device), and can achieve spatial position synchronization, interaction event recognition, and tactile feedback consistency control among multiple virtual learners.

[0065] A method for enhancing VR teaching experience, comprising the following specific steps:

[0066] Step S1: Collecting position data and posture data of multiple virtual reality trainees in a virtual reality environment.

[0067] In this embodiment, for the virtual reality collaborative teaching scenario, the position data and posture data of multiple virtual reality trainees in the virtual reality environment are collected by deploying a spatial tracking module on the devices worn by the trainees.

[0068] Specifically, the spatial tracking module includes but is not limited to: infrared positioning markers on the head-mounted display device (HMD), the IMU inertial measurement unit, the accelerometer and gyroscope embedded in the handle, and the camera visual tracking system. The camera visual tracking system is based on the optical / inertial / visual fusion positioning algorithm to calculate the spatial status information of each trainee in real time.

[0069] In actual deployment, the virtual reality terminal devices of the trainees collect the following two types of data at a fixed sampling frequency (such as 90Hz): position data, including the three-dimensional coordinate values ​​(X, Y, Z) of the center of the head and the centers of the left and right palms, in meters; posture data, including the direction information of the above-mentioned parts, preferably expressed in the unitized quaternion (Qx, Qy, Qz, Qw) format, where Qx, Qy, Qz represent the rotation axis vector components, and Qw represents the cosine value of the rotation angle component.

[0070] Step S2: Based on the position data and posture data of multiple trainees, a multi-user coordinated interaction model is constructed, where the multi-user coordinated interaction model is used to fuse the spatial positioning information of the multiple trainees and identify the interaction relationships and corresponding interaction events between the trainees.

[0071] like Figure 2 As shown in the figure, multiple trainees in the virtual teaching system are in different positions in the virtual space, and their positioning and posture are represented by spatial state vectors. The rectangular area on the left side of the figure represents the distribution of trainees in the virtual scene, the circle plus two lines icon represents the trainees, and the small circle mark indicates the location where the interaction event is detected.

[0072] The specific steps of step S2 are:

[0073] Step S201: performing time stamp synchronization processing on the position data and posture data of the trainee in the VR environment;

[0074] Specifically, the server is used as the global time reference, and the current time is synchronized to all terminals through UDP low-latency broadcast or PTP (Precision Time Protocol) protocol. After receiving the time synchronization instruction, the terminal adjusts the local data collection time to a unified reference timestamp, and uniformly processes the timestamps of the trainees' position data and posture data in the VR environment.

[0075] Step S202: Based on the processed position data and posture data of the trainee in the VR environment, three-dimensional coordinate mapping and posture calculation are performed on the spatial position information of the trainee, and a spatial state vector of the trainee is constructed.

[0076] The specific steps of step S202 are:

[0077] Step S2021: Convert the position data of the trainee in the VR environment into three-dimensional coordinate values ​​in the virtual environment, and the conversion is performed based on a preset coordinate mapping matrix or calibration parameters.

[0078] Specifically, during the initialization stage of the teaching scene, the relative position and orientation of each device are obtained through fixed reference calibration points, such as the scene origin or a known reference device, and a mapping matrix corresponding to each device is established. It should be noted that if the device installation position is fixed, the corresponding preset calibration parameter file can be directly loaded according to the device number without the need to rebuild the mapping matrix in real time. The preset coordinate mapping matrix and offset vector are used to map the original coordinates to the global coordinates of the virtual scene.

[0079] Step S2022: Fusing the three-dimensional coordinate values ​​with the posture data of the trainee in the VR environment to construct a spatial state vector of the trainee at the current frame time, where the spatial state vector is a combination vector of the position vector and the posture vector.

[0080] In this embodiment, the processed three-dimensional coordinate values ​​are spliced ​​and combined with the collected posture information to form a complete spatial state vector. It should be noted that in a virtual reality interactive environment, the position information alone can only reflect where the trainee is, while the posture information further characterizes the direction. Therefore, the fused spatial state vector can not only describe the spatial occupancy, but also be used to judge the interaction intention, line of sight direction and action path; for example, the spatial state vector [X, Y, Z, Qx, Qy, Qz, Qw], where (X, Y, Z) represents the three-dimensional coordinate value and (Qx, Qy, Qz, Qw) represents the collected posture information, which is a quaternion. This vector describes the position information and orientation information of the trainee at the current moment.

[0081] By constructing a spatial state vector, the spatial behavior of trainees is modeled, which can be used for subsequent distance calculation, orientation analysis, and motion trajectory recognition.

[0082] Step S2023: normalize the spatial state vector.

[0083] In this embodiment, the normalization processing includes: quaternion unitization processing, position vector limiting and filtering, and interpolation frame filling processing. The quaternion unitization processing is to calculate the module length of the original quaternion, and then divide each number by the module length so that the sum of the squares of the four numbers is 1; the position vector limiting and filtering includes: limiting processing to determine whether the moving distance of a single frame exceeds the speed at which humans can perform actions. If it exceeds, it is regarded as a jump and correction is performed, denoising and boundary constraints; interpolation frame filling processing performs linear interpolation or spherical interpolation according to the position and posture of the previous and next frames to fill in the missing frames.

[0084] Step S203: Based on the spatial state vector, the relative position relationship and interaction action trajectory between the trainees are analyzed to determine whether there are conditions for an interaction event to occur between the trainees.

[0085] The specific steps of step S203 are:

[0086] Step S2031: Compare the spatial state vectors of any two or more trainees in the same time frame, and calculate the spatial distance, relative orientation angle, and posture similarity between the trainees.

[0087] In this embodiment, at any time frame t, the spatial state vector set S(t) of all trainees is obtained, S(t)={S1(t), S2(t), ..., S n (t)}, where n represents the number of trainees, S n (t) represents the spatial state vector of the nth trainee, S n (t)=[X n ,Y n ,Z n ,Qx n ,Qy n ,Qz n ,Qw n ],(X n ,Y n ,Z n ) represents the position vector of the nth trainee, (Qx n ,Qy n ,Qz n ,Qw n ) represents the attitude vector in normalized quaternion form.

[0088] Specifically, spatial distance is used to evaluate the relative distance between two trainees. The specific formula is: , D ij Represents the distance between trainee i and trainee j. The smaller the value, the closer trainees i and j are in the scene, i≤n.

[0089] The relative heading angle converts the attitude vector in the form of a unit quaternion into a unit direction vector. The conversion method is obtained by quaternion-rotation matrix transformation. The specific formula is: ,in represents the unit direction vector of trainee i, Represents the attitude vector of trainee i in unitized quaternion form, Represents the initial default forward vector, such as [0,0,1] means the Z axis is facing forward, and calculates the angle between the direction vectors of trainees i and j. , the formula is: , arccos() represents the inverse cosine function, Indicates vector modulus, v j Represents the unit direction vector of trainee j.

[0090] Posture similarity is used to determine whether the direction and structure of the movements performed by two trainees are consistent. The specific formula is: ,in, Indicates the posture similarity between trainee i and trainee j.

[0091] Step S2032: judging whether the spatial distance is lower than a first distance threshold, whether the relative orientation angle is within a preset angle range, and whether the posture similarity exceeds an action matching threshold based on a preset interaction determination threshold;

[0092] It should be noted that in multi-person VR collaborative teaching scenarios, the spatial relationship and movement intentions between trainees vary depending on the course content, and the specific threshold settings are also set according to the actual course.

[0093] Step S2033: If the preset interaction determination threshold is met, the change trend of the trainee's motion trajectory in the adjacent time window is analyzed, and the motion trajectory is extracted based on the time series composed of continuous space state vectors.

[0094] In this embodiment, state vectors of several frames forward and backward are extracted based on the current time frame to form a time window of fixed length for motion trajectory analysis. Specific parameters include: a time window length of 1 second and a sampling interval of 20 milliseconds, and the window contains 50 frames of continuous spatial state vectors.

[0095] Furthermore, the trajectory of the trainee within the window is represented as a set of state vector sequences arranged in chronological order, which contains dual information of position and posture.

[0096] The specific motion trajectory change trends include relative position trends, posture evolution trends and trajectory shape trends.

[0097] Step S2034: Match the motion trajectory with a predefined interaction event pattern library. If the matching result reaches a set confidence threshold, it is determined that an interaction event occurs between the trainees.

[0098] The benefits of this step are: it can identify dynamic changes in behavior within a short period of time, and construct more distinctive behavioral features by fusing trajectory morphology, direction trend and posture evolution.

[0099] Step S204: When it is detected that the interaction action between two or more trainees meets the preset collaborative interaction rules, it is determined to be a type of interaction event, and the interaction event is classified and processed.

[0100] Furthermore, if Figure 2 As shown in Figure 2, when it is recognized that the spatial states between some trainees meet the collaborative interaction conditions, such as close relative positions and consistent posture directions, the relevant trainees are classified into the same interaction group and Figure 2 It is marked with a dotted rectangle.

[0101] exist Figure 2 In the right area, the spatial state vectors of the group of trainees at the moment the interaction event is triggered are extracted and marked as spatial state vector 1 to spatial state vector 4. These state vectors include the three-dimensional coordinates and posture vectors of each trainee.

[0102] Step S3: Generate tactile feedback data corresponding to the interaction event according to the interaction event. The specific steps of step S3 are:

[0103] Step S301: According to the interaction event, a tactile feedback template matching the interaction event type is selected from a preset tactile feedback template library, where the tactile feedback template includes vibration frequency, amplitude, duration, and feedback mode.

[0104] In this embodiment, the preset tactile feedback template library includes at least the following typical interaction event types and their corresponding tactile feedback parameter templates: contact events, such as virtual handshakes, touching objects, tool handovers, etc.; collaborative events, such as jointly carrying virtual objects, synchronously operating instrument knobs, etc.; status events, such as operation success or failure prompts, dangerous area warnings, etc.

[0105] Step S302: Calculate the trainee's tactile feedback position, direction, and tactile intensity parameters to generate personalized adjustment feedback data.

[0106] On the one hand, in this embodiment, the center position of the interaction event is first determined, such as the position of the virtual device involved in a collaborative assembly task. Then, based on the spatial state vector of the trainee at that moment, the body part or controller port closest to the interaction center point is identified. For example: if the interaction object is a virtual button, and the right hand of a trainee is near the button and facing the interaction area, the tactile feedback position is determined to be the right palm.

[0107] On the other hand, the action vector of the tactile feedback is calculated based on the action direction of the interaction event and the spatial orientation of the trainees.

[0108] The feedback intensity value is determined based on the level of the interaction event, the distance between the trainee and the event center point, whether the trainee's action is dominant or passive, and user preference parameters, and then personalized adjustment feedback data is generated.

[0109] Step S303: Based on the tactile feedback template and the personalized adjustment feedback data, a tactile feedback data packet for the trainee is generated, and the data packet includes: feedback type, target device number, control instruction format, execution timing information, etc.

[0110] Step S4: performing calibration processing on the tactile feedback data, adjusting the tactile feedback data based on the response delay and spatial position difference of the VR device of the trainee, and synchronously sending the calibrated tactile feedback data to the VR device of the trainee.

[0111] like Figure 3 As shown, after identifying the interaction event and extracting the spatial state vector of the trainee, firstly, spatial state vectors 1 to 4 are constructed for the group of trainees respectively, and they are used to calculate the spatial propagation difference and the tactile feedback time offset value.

[0112] The specific steps of step S4 are:

[0113] Step S401: Acquire the trainee's tactile feedback response parameters, which include data reception delay, drive execution delay, and device triggering delay.

[0114] In this embodiment, the data reception delay is evaluated by the timestamp synchronization method; the drive execution delay is reported by the device operating system or operating environment; and the device trigger delay is obtained by the factory calibration value + user usage data feedback correction method.

[0115] Step S402: Calculating the spatial propagation difference between the trainees based on their spatial state vectors at the time of the interaction event. The spatial propagation difference is the relative propagation time difference of the tactile feedback data in the virtual scene.

[0116] The specific steps of step S402 are:

[0117] Step S4021: Determine the trigger center point of the interaction event, where the trigger center point is calculated based on the spatial coordinates of all trainees.

[0118] In this embodiment, based on the list of trainees participating in the interactive event, the spatial position coordinates of each trainee at the triggering moment are obtained, and the spatial coordinates of all interactive event participants are weighted averaged to obtain a point at the geometric center as the triggering center point of the interactive event.

[0119] Step S4022: Calculate the distance between each trainee and the trigger center point of the interaction event, and set it as the propagation path length.

[0120] Step S4023: Based on the propagation path length and the simulated propagation speed of the tactile signal in the VR scene, the relative propagation delay of the trainee is calculated.

[0121] Step S4024: Taking the minimum value of all relative propagation delays as a reference, calculate the propagation difference of the trainee relative to the trainee who responds earliest, which is the spatial propagation difference.

[0122] This step, based on the participant's spatial state vector at the moment the interaction event is triggered, sequentially calculates the interaction center point and the propagation path length from the participant to that center point. Combined with the preset tactile signal propagation speed, the relative propagation delay for each participant is inferred. Using the shortest propagation time as a benchmark, the propagation difference for each participant is then calculated, representing the spatial propagation difference. This step simulates the transmission of tactile signals in virtual space, effectively quantifying the perceived time differences between participants due to their different spatial positions, providing a precise basis for subsequent tactile feedback timing calibration.

[0123] Step S403: fusing the trainee's tactile feedback response parameter with the spatial propagation difference to calculate the trainee's tactile feedback time offset value.

[0124] Step S4031: Calculate the total feedback delay of the trainee, where the total feedback delay is obtained by weighted summation of communication delay, device execution delay, and spatial propagation delay.

[0125] Step S4032: selecting the minimum value of all total feedback delays as a reference time point for global tactile feedback.

[0126] Step S4033: The difference between the total feedback delay of the trainee and the reference time point of the global tactile feedback is used as the tactile feedback time offset value of the trainee.

[0127] This step calculates a unified total feedback delay by integrating the tactile feedback response parameters of each trainee, including communication delay, device execution delay, etc., with their spatial propagation differences. The minimum total delay value among all trainees is used as the reference time point for global tactile feedback, and the time offset value of each trainee is further obtained. This offset value is used to dynamically adjust the execution timing of the tactile feedback, thereby achieving synchronized tactile perception of multiple trainees on different devices and in different spatial positions, thereby significantly improving the virtual teaching experience.

[0128] Step S404: adjusting the execution timing information in the trainee's tactile feedback data packet according to the time offset value, so that the trainee can synchronously trigger the tactile feedback at the target time point.

[0129] like Figure 3 As shown, in Figure 3 The left part of represents the spatial position and posture information of multiple trainees when the interaction event occurs. Based on the spatial state vector of each trainee, the spatial propagation path length and propagation delay relative to the center point of the interaction event are calculated to obtain the spatial propagation difference.

[0130] Furthermore, if Figure 3As shown, the spatial propagation difference of each trainee is fused with its local tactile feedback response parameters, which include communication delay and device execution delay, to obtain the tactile feedback time offset value of each trainee.

[0131] The calculation of spatial propagation difference and tactile feedback time offset value is completed in sequence, and the tactile feedback trigger timing of each trainee's equipment is adjusted through the compensation processing mechanism.

[0132] Through the above processing, tactile feedback is uniformly executed with compensated values ​​in multiple trainee terminal devices, ensuring that all trainees have consistent tactile experience in time, direction, and intensity for the same interactive event in the virtual environment, thereby significantly improving the learning experience in collaborative teaching.

[0133] Example 2

[0134] See also Figure 4 , another embodiment provided by the present invention: a VR teaching experience enhancement system, comprising: a data acquisition module, an interaction recognition module, a tactile feedback generation module, and a calibration synchronization module;

[0135] The data acquisition module is used to collect the position data and posture data of the trainees in the VR environment;

[0136] The interaction recognition module builds a multi-user coordinated interaction model based on the position data and posture data of the trainees in the VR environment;

[0137] The tactile feedback generating module generates tactile feedback data corresponding to the interaction event based on the interaction event;

[0138] The calibration and synchronization module is used to perform calibration processing on the tactile feedback data and synchronously send the calibrated tactile feedback data to the VR device of the trainee.

[0139] The calibration synchronization module includes: a response parameter acquisition unit, a propagation time difference calculation unit and a calibration adjustment unit;

[0140] The response parameter acquisition unit is used to obtain the tactile feedback response parameters of the trainees;

[0141] The propagation time difference calculation unit calculates the spatial propagation difference between the trainees based on the spatial state vectors of the trainees at the time of the interaction event;

[0142] The calibration adjustment unit is used to fuse the tactile feedback response parameters of the trainee with the spatial propagation difference, calculate the tactile feedback time offset value of the trainee, and adjust the execution timing information in the tactile feedback data packet of the trainee according to the time offset value, so that the trainee can synchronously trigger the tactile feedback at the target time point.

[0143] In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0144] The above-described specific embodiments further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for enhancing VR teaching experience, characterized in that: include: Collect the position and posture data of trainees in the VR environment; Based on the position data and posture data of the trainees in the VR environment, a multi-user coordinated interaction model is constructed, wherein the multi-user coordinated interaction model is used to integrate the spatial positioning data and posture data of the trainees and identify the interaction relationships and interaction events between the trainees; Based on the interaction event, generating tactile feedback data corresponding to the interaction event; Performing calibration processing on the tactile feedback data, and synchronously sending the calibrated tactile feedback data to the trainee's VR device, the calibration processing including adjusting the tactile feedback data based on the trainee's spatial propagation difference and the tactile feedback time offset value; The performing calibration processing on the tactile feedback data includes: Acquiring tactile feedback response parameters of the trainee, wherein the response parameters include data reception delay, drive execution delay, and device triggering delay; Calculating the spatial propagation difference between the trainees based on their spatial state vectors at the time of the interaction event, where the spatial propagation difference is the relative propagation time difference of the tactile feedback data in the virtual scene; The tactile feedback response parameter of the trainee is integrated with the spatial propagation difference to calculate the tactile feedback time offset value of the trainee; Adjusting the execution timing information in the trainee's tactile feedback data packet according to the time offset value so that the trainee can synchronously trigger the tactile feedback at the target time point; The calculation of the spatial propagation difference between the trainees based on the spatial state vector of the trainees at the time of the interaction event, where the spatial state vector is a combination vector of the position vector and the posture vector, includes: Determine a trigger center point of the interaction event, where the trigger center point is calculated based on the spatial coordinates of all trainees; Calculate the distance between each trainee and the trigger center point of the interaction event and set it as the propagation path length; Based on the propagation path length and the simulated propagation speed of the tactile signal in the VR scene, the relative propagation delay of the trainees is calculated; Taking the minimum value of all relative propagation delays as a reference, the propagation difference of the trainee relative to the earliest responding trainee is calculated, which is the spatial propagation difference.

2. A method for enhancing VR teaching experience according to claim 1, characterized in that: The multi-user coordinated interaction model is constructed based on the position data and posture data of the trainees in the VR environment, including: Perform time stamp synchronization on the position and posture data of trainees in the VR environment; Based on the processed position and posture data of the trainees in the VR environment, the three-dimensional coordinate mapping and posture calculation of the trainees' spatial position information are performed, and the spatial state vector of the trainees is constructed; Based on the spatial state vector, the relative position relationship and interaction movement trajectory between the trainees are analyzed to determine whether there are conditions for an interaction event to occur between the trainees; When it is detected that the interaction actions between two or more trainees meet the preset collaborative interaction rules, it is determined to be a type of interaction event, and the interaction event is classified and processed.

3. A method for enhancing VR teaching experience according to claim 2, characterized in that: The method of performing three-dimensional coordinate mapping and posture calculation on the spatial position information of the trainee based on the processed position data and posture data of the trainee in the VR environment, and constructing the spatial state vector of the trainee, includes: Converting the position data of the trainee in the VR environment into three-dimensional coordinate values ​​in the virtual environment, wherein the conversion is performed based on a preset coordinate mapping matrix or calibration parameters; The three-dimensional coordinate values ​​are integrated with the posture data of the trainee in the VR environment to construct the spatial state vector of the trainee at the current frame time. The spatial state vector is a combination vector of the position vector and the posture vector; The spatial state vector is normalized.

4. A method for enhancing VR teaching experience according to claim 3, characterized in that: The analyzing the relative position relationship and interaction trajectory between the trainees based on the spatial state vector to determine whether conditions for an interaction event to occur exist between the trainees includes: Compare the spatial state vectors of any two or more trainees in the same time frame and calculate the spatial distance, relative orientation angle and posture similarity between the trainees; According to a preset interaction determination threshold, determining whether the spatial distance is lower than a first distance threshold, whether the relative orientation angle is within a preset angle range, and whether the posture similarity exceeds an action matching threshold; If the preset interaction determination threshold is met, the trend of the trainee's motion trajectory in the adjacent time window is analyzed. The motion trajectory is extracted based on the time series composed of continuous spatial state vectors. The motion trajectory is matched with a predefined interaction event pattern library. If the matching result reaches a set confidence threshold, it is determined that an interaction event has occurred between the trainees.

5. A method for enhancing VR teaching experience according to claim 4, characterized in that: The generating, based on the interaction event, tactile feedback data corresponding to the interaction event includes: According to the interaction event, a tactile feedback template matching the interaction event type is selected from a preset tactile feedback template library, wherein the tactile feedback template includes vibration frequency, amplitude, duration, and feedback mode; Calculate the trainee's tactile feedback position, direction, and tactile intensity parameters to generate personalized adjustment feedback data; Based on the tactile feedback template and the personalized adjustment feedback data, a tactile feedback data packet for the trainees is generated, and the data packet includes: feedback type, target device number, control instruction format, and execution timing information.

6. A method for enhancing VR teaching experience according to claim 5, characterized in that: The step of fusing the trainee's tactile feedback response parameter with the spatial propagation difference to calculate the trainee's tactile feedback time offset value includes: Calculating a total feedback delay of the trainee, where the total feedback delay is obtained by weighted summation of communication delay, device execution delay, and spatial propagation delay; The minimum value of all total feedback delays is selected as the reference time point of global tactile feedback; The difference between the total feedback delay of the trainee and the reference time point of the global tactile feedback is used as the tactile feedback time offset value of the trainee.

7. A VR teaching experience enhancement system, used to implement a VR teaching experience enhancement method according to any one of claims 1 to 6, characterized in that: include: Data acquisition module, interactive recognition module, tactile feedback generation module, calibration and synchronization module; The data acquisition module is used to collect the position data and posture data of the trainees in the VR environment; The interaction recognition module builds a multi-user coordinated interaction model based on the position data and posture data of the trainees in the VR environment; The tactile feedback generating module generates tactile feedback data corresponding to the interaction event based on the interaction event; The calibration and synchronization module is used to perform calibration processing on the tactile feedback data and synchronously send the calibrated tactile feedback data to the VR device of the trainee.

8. A VR teaching experience enhancement system according to claim 7, characterized in that: The calibration synchronization module includes: a response parameter acquisition unit, a propagation time difference calculation unit and a calibration adjustment unit; The response parameter acquisition unit is used to obtain the tactile feedback response parameters of the trainees; The propagation time difference calculation unit calculates the spatial propagation difference between the trainees based on the spatial state vectors of the trainees at the time of the interaction event; The calibration adjustment unit is used to fuse the tactile feedback response parameters of the trainee with the spatial propagation difference, calculate the tactile feedback time offset value of the trainee, and adjust the execution timing information in the tactile feedback data packet of the trainee according to the time offset value, so that the trainee can synchronously trigger the tactile feedback at the target time point.

Images