A method for driving a somatosensory device based on a VR scene

Abstract

The application discloses a VR scene-based somatosensory device driving method, which is used in the somatosensory interaction field and includes the following steps: dividing into fire scene paragraphs and fire extinguishing paragraphs, and defining a mapping table; constructing a trigger mechanism and a double queue based on the mapping table, judging the paragraph or picture trigger event entered by an experimenter, and sending corresponding somatosensory instructions into a normal queue or a low-delay queue in the double queue; for the heat wave pulse instructions in the normal queue, calculating a dynamic delay value based on a visual rendering pipeline, and driving a somatosensory working thread to execute the heat wave pulse instructions after sleeping for the dynamic delay value; for the spraying instructions in the low-delay queue, setting a preemptive scheduling priority, interrupting the heat wave pulse instructions being executed by the somatosensory working thread and preferentially executing the spraying instructions. The application realizes accurate time synchronization of the heat wave somatosensory and the flame picture, and guarantees instant response of the fire extinguishing spraying action.

Description

Technical Field

[0001] This application relates to the field of motion-sensing interaction, and more particularly to a motion-sensing device driving method based on a VR scene. Background Technology

[0002] VR large space is a new form of presentation based on virtual reality technology that is currently emerging both domestically and internationally. By showcasing people and events in a virtual world within a specific space, it has broad application prospects in fields such as cultural tourism, film and television industry, cultural creativity, and cultural dissemination and inheritance.

[0003] VR large-space experience venues are typically set up in flat indoor spaces of 200 to 300 square meters. Experiencers wear VR glasses or VR headsets and move forward according to built-in guidance, with corresponding virtual scenes appearing in their field of vision. When a fire scene appears in the virtual scene, the experiencer can also feel the scorching heat on site. Similarly, when the virtual scene shows firefighters extinguishing a fire, the experiencer can also feel the sporadic water mist or droplets sprayed on site. This design achieves the experience effects of augmented reality and mixed reality. To achieve this, it is necessary to realize two-way linkage between the virtual scene and the on-site experience device to achieve a high degree of consistency between the virtual image and the on-site physical sensation.

[0004] In existing VR fire-fighting motion-sensing solutions, the independent operation of vision, heat, water mist, and audio has inherent time delays, which can easily lead to asynchrony between motion and vision, causing sensory dysfunction and dizziness. Using only fixed time delay compensation cannot adapt to dynamic loads, and there are also problems such as scene motion matching disorder, disordered command scheduling, delayed sprinkler response, no task priority, and difficulty in recovery after interruption.

[0005] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0006] To overcome the above problems, this application aims to propose a motion-sensing device driving method based on VR scenes. The purpose is to solve the problems in existing VR fire-fighting motion-sensing solutions, where the independent operation of vision, heat, water mist, and audio has inherent time delays, which easily leads to asynchrony between motion sensing and vision, causing sensory dysfunction and dizziness; fixed time delay compensation alone cannot adapt to dynamic loads, and there are also technical problems such as scene motion matching disorder, disordered command scheduling, delayed sprinkler response, no task priority, and difficulty in recovery after interruption.

[0007] Therefore, the specific technical solution adopted in this application is as follows: A method for driving a motion-sensing device based on a VR scene, the method comprising: S1. In the VR headset, the VR large space is divided into fire scene segment and fire extinguishing segment, and a mapping table of plot screen and somatosensory trigger events is defined for each segment. S2. Based on the mapping table, a triggering mechanism and dual queues are constructed. The triggering mechanism determines the segment or scene that the user enters based on the mapping table and the plot screen currently being played by the VR headset, and sends the corresponding haptic commands to the regular queue or low-latency queue in the dual queues. S3. For heat wave pulse instructions in the regular queue, calculate the dynamic delay value based on the visual rendering pipeline, and drive the motion sensing worker thread to sleep according to the dynamic delay value before executing the heat wave pulse instruction. S4. For the spraying instructions in the low-latency queue, set a preemptive scheduling priority, so that the main control thread interrupts the heat wave pulse instruction being executed by the somatosensory working thread and executes the spraying instruction first. S5. Record the expected visual timestamp and the actual hardware execution time of each motion-sensing command, and use the deviation between the two as the basis for adjusting the dynamic latency value and the preemptive scheduling priority.

[0008] Optionally, a mapping table between scene images and motion-triggered events is defined for each paragraph, including: In VR headsets, the physical space within the VR large space is divided into fire scene segments and fire extinguishing segments according to the scenario and plot. Design VR scene sequences for the fire scene segment that include flames burning and heat wave radiation, and define heat wave pulse haptic trigger events based on the scene sequences of the fire scene segment; Design VR scene scenes for the fire extinguishing segment that include firefighters spraying water to extinguish fires, and define spraying motion trigger events based on the scene scenes of the fire extinguishing segment; By binding fire scene segments with heat wave pulse events and fire extinguishing segments with sprinkler events, a mapping table is formed of segments, plot scenes, and motion-triggered events.

[0009] Optionally, the corresponding motion-sensing command is sent to either the regular queue or the low-latency queue in the dual queue, including: An event detection mechanism is built based on a mapping table. This mechanism is used to monitor events in real time, such as when a user enters a segment or when a VR headset plays a specific scene. Create a dual queue consisting of a regular queue and a low-latency queue, and use the dual queue to store motion-sensing commands including heat wave pulse commands and spray commands; The detection result of the paragraph is called and the triggering mechanism queries the mapping table to determine the corresponding motion control command for the current paragraph. Based on the type of motion control command, the motion control command is sent to the corresponding queue; If it is a heat wave pulse command, it will be sent to the regular queue; If it is a spray command, it is sent to the low-latency queue.

[0010] Optionally, the triggering mechanism queries the mapping table to determine the motion-sensing command corresponding to the current paragraph, including: Based on the current spatial location of the participant, determine whether the participant is in the fire scene section or the fire extinguishing section, and obtain the section name and determination result; The trigger mechanism retrieves the paragraph name and judgment result, and uses them as the basis for querying the mapping table; The mapping table is queried based on the paragraph name to match the heat wave pulse command corresponding to the fire scene paragraph or the sprinkler command corresponding to the fire extinguishing paragraph; The trigger mechanism outputs the matching result to determine whether the current paragraph corresponds to a heat wave pulse command or a spray command.

[0011] Optionally, the formula for calculating the dynamic delay value is: In the formula, Indicates the dynamic delay value of the motion-sensing command; Indicates the baseline latency for VR visual rendering; This represents the queue load weighting coefficient; This represents the weighting coefficient based on the queue length. This indicates the number of heatwave instructions currently pending execution in the regular queue; Indicates the maximum number of instructions that a regular queue can hold; This represents the weighting coefficient for the average instruction waiting time. This represents the average waiting time for all instructions awaiting execution in the current queue; This represents the maximum allowed waiting time threshold for a single motion-sensing command in the queue. This indicates the weighting coefficient for emergency orders; This indicates the number of emergency heatwave instructions currently in the queue; This represents the CPU load weighting coefficient; Indicates GPU utilization; This represents the GPU load weighting coefficient; This indicates CPU utilization (similar to UC, but needs to be distinguished). This indicates the inherent response latency of motion-sensing hardware; Indicates the number of historical instruction samples; Indicates the first One historical motion-sensing command; Indicates the first The actual hardware execution timestamp of each instruction; Indicates the first The expected visual timestamp of the instruction.

[0012] Optionally, the method for driving the motion-sensing worker thread to sleep after a dynamic delay value and then execute the heat wave pulse command is as follows: The motion sensing worker thread acquires dynamic latency values ​​including visual rendering baseline latency, queue load prediction values, CPU and GPU load prediction values, hardware inherent latency, and historical deviation feedback values. The sleep duration is determined based on the dynamic latency value, and the user's state is set to sleep, entering a waiting state. After the hibernation period ends, the motion sensing worker thread is awakened, immediately sends an execution command to the heat wave pulse device, and records the actual hardware execution timestamp of the heat wave pulse command.

[0013] Optionally, the main control thread may interrupt the heat wave pulse instruction being executed by the motion sensing worker thread and prioritize the execution of the spray instruction, including: Assign a preemptive scheduling priority to the spraying instructions in the low-latency queue, and the preemptive scheduling priority is higher than the scheduling priority of all instructions in the regular queue; The main control thread monitors the low-latency queue status in real time. Once a spray command is detected to be enqueued, the execution status of the current motion-sensing worker thread is immediately obtained. Based on the execution status of the motion sensing worker thread, if it is detected that a heat wave pulse command is being executed, an interrupt signal is sent to the motion sensing worker thread and the execution of the heat wave pulse command is terminated. After the main control thread takes over the execution, it prioritizes the execution of the spray command. When the execution is completed, it sends a recovery signal to the motion sensing worker thread, allowing the motion sensing worker thread to continue processing the interrupted heat wave pulse command.

[0014] Optionally, sending an interrupt signal to the motion-sensing worker thread and terminating the execution of the heat wave pulse command includes: Based on preemptive scheduling priority, the type of currently executed somatosensory instruction is dynamically compared with the priority of the enqueued spray instruction to determine whether preemption is triggered. When the spray command has a higher priority than the heat wave pulse command, the main control thread immediately interrupts the current command and generates a preemptive interrupt signal; The main control thread sends a preemption interrupt signal to the motion sensing worker thread. The motion sensing worker thread immediately suspends the current task at any point during instruction execution and saves the status information of the interrupted instruction. The main control thread confirms that the motion sensing worker thread has been suspended and switches execution control from the motion sensing worker thread to the main control thread.

[0015] Optionally, the deviation between the two can be used as the basis for adjusting the dynamic delay value and the preemptive scheduling priority, including: When executing each motion-sensing command, the motion-sensing worker thread records the expected visual timestamp corresponding to the command, and records the actual hardware execution time after the motion-sensing command is completed. Obtain the expected visual timestamp and the actual hardware execution time of the same instruction, and calculate the difference between the expected visual timestamp and the actual hardware execution time as the execution deviation value of the haptic instruction; The execution deviation value is stored in the deviation queue, and a preset number of the latest instruction deviation value samples are maintained according to the first-in-first-out principle; Read the deviation value samples in the deviation queue, calculate the weighted average of all deviation value samples, and use the weighted average as the basis for adjusting the dynamic delay value and the preemptive scheduling priority.

[0016] Compared with the prior art, this application has the following beneficial effects: 1. This application divides a large VR space into fire scene and fire extinguishing sections and binds them to a motion mapping table. Combined with dual-queue hierarchical scheduling, dynamic delay calculation based on visual rendering, preemptive interruption mechanism for sprinkler commands, and adaptive adjustment based on motion deviation feedback, it achieves precise time synchronization between heat wave sensation and flame imagery, and ensures immediate response of fire extinguishing sprinkler actions. This enhances the immersiveness and realism of the VR firefighting experience, as well as the stability and adaptability of the system operation.

[0017] 2. This application solves the problems of mismatched scene and motion sensing and disordered instruction scheduling by dividing VR large space segments and constructing a plot and motion sensing mapping table, combined with dual queue classification scheduling and dual detection of spatial position and screen content, thereby improving the accuracy of motion sensing triggering and scheduling order.

[0018] 3. This application adopts multi-dimensional dynamic delay calculation to replace fixed delay compensation, which can adapt to dynamic system load, accurately match the synchronization sequence of heat wave sensation and VR screen, avoid the visual misalignment caused by inherent delay, and ensure reasonable rhythm of sensation output.

[0019] 4. This application solves the problems of delayed response of sprinkler action, lack of priority of sonic tasks and difficulty in recovery after interruption by setting high preemption priority of sprinkler command and complete interruption suspension and recovery mechanism, so as to ensure that fire extinguishing sonic response is timely and the task is not lost. Attached Figure Description

[0020] The above-mentioned features, characteristics, and advantages of this application, as well as their implementation methods, will become clearer and more understandable in conjunction with the following description of the embodiments, which are illustrated in detail with reference to the accompanying drawings. Schematic diagrams are shown here: Figure 1 This is a flowchart of the motion-sensing device driving method in this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0022] According to embodiments of this application, a method for driving motion-sensing devices based on VR scenes is provided. By dividing a large VR space into fire scene and fire extinguishing segments and binding them to a motion-sensing mapping table, and combining dual-queue hierarchical scheduling, dynamic latency calculation based on visual rendering, a preemptive interruption mechanism for sprinkler commands, and adaptive adjustment based on motion-sensing deviation feedback, this method achieves precise time synchronization between the heat wave sensation and the flame image, while ensuring the immediate response of fire extinguishing sprinkler actions. This enhances the immersiveness and realism of the VR firefighting experience, as well as the stability and adaptability of the system operation. Figure 1 As shown, the method includes: S1. In the VR headset, the VR large space is divided into fire scene segment and fire extinguishing segment, and a mapping table of plot scenes and somatosensory trigger events is defined for each segment.

[0023] Preferably, a mapping table is defined for each paragraph, showing the scene and the motion-triggered events, including: In VR headsets, the physical space within the VR large space is divided into fire scene segments and fire extinguishing segments according to the scenario and plot. Design VR scene sequences for the fire scene segment that include flames burning and heat wave radiation, and define heat wave pulse haptic trigger events based on the scene sequences of the fire scene segment; Design VR scene scenes for the fire extinguishing segment that include firefighters spraying water to extinguish fires, and define spraying motion trigger events based on the scene scenes of the fire extinguishing segment; By binding fire scene segments with heat wave pulse events and fire extinguishing segments with sprinkler events, a mapping table is formed of segments, plot scenes, and motion-triggered events.

[0024] It needs to be explained that in VR headsets, based on the scenarios and plot logic of the virtual world, the physical VR space is divided into two functional areas: the fire scene corresponds to the scenario of a fire and heat radiation; the fire extinguishing scene corresponds to the scenario of firefighters appearing and water cannons extinguishing the fire. The VR scene of burning flames and radiating heat waves was designed as the visual presentation of this segment; based on these scenes, a heat wave pulse haptic trigger event was defined, that is, when the scene of burning flames was played, a heat wave pulse command was automatically generated. The design incorporates VR scenes of firefighters using water sprayers to extinguish fires as the visual presentation of this segment; based on the fire extinguishing scenes, a spraying motion trigger event is defined, that is, when the fire spraying scene is played, a spraying command is automatically generated; The fire scene segment forms a mapping with its corresponding flame burning images and heat wave pulse events; the fire extinguishing segment forms another mapping with its corresponding water spray gun fire extinguishing images and spraying events. The resulting mapping table serves as the core data index for the entire driving method.

[0025] S2. Based on the mapping table, a triggering mechanism and dual queues are constructed. The triggering mechanism determines the segment or scene that the user enters based on the mapping table and the plot screen currently being played on the VR headset, and sends the corresponding motion control command to the regular queue or low-latency queue in the dual queues.

[0026] Preferably, the corresponding motion-sensing command is sent to either the regular queue or the low-latency queue in the dual queues, including: An event detection mechanism is built based on a mapping table. This mechanism is used to monitor events in real time, such as when a user enters a segment or when a VR headset plays a specific scene. Create a dual queue consisting of a regular queue and a low-latency queue, and use the dual queue to store motion-sensing commands including heat wave pulse commands and spray commands; The detection result of the paragraph is called and the triggering mechanism queries the mapping table to determine the corresponding motion control command for the current paragraph. Based on the type of motion control command, the motion control command is sent to the corresponding queue; If it is a heat wave pulse command, it will be sent to the regular queue; If it is a spray command, it is sent to the low-latency queue.

[0027] Preferably, the triggering mechanism queries the mapping table to determine the motion-sensing command corresponding to the current paragraph, including: Based on the current spatial location of the participant, determine whether the participant is in the fire scene section or the fire extinguishing section, and obtain the section name and determination result; The trigger mechanism retrieves the paragraph name and judgment result, and uses them as the basis for querying the mapping table; The mapping table is queried based on the paragraph name to match the heat wave pulse command corresponding to the fire scene paragraph or the sprinkler command corresponding to the fire extinguishing paragraph; The trigger mechanism outputs the matching result to determine whether the current paragraph corresponds to a heat wave pulse command or a spray command.

[0028] It should be explained that the triggering mechanism is based on two criteria: first, the paragraph information defined in the mapping table, and second, the scene currently being played on the VR headset. The triggering mechanism determines the trigger source type (segment entry or scene playback) based on these two criteria and sends the corresponding motion control command to the regular queue or the low-latency queue in the dual queues. An event detection mechanism is built based on a mapping table. This mechanism monitors two types of triggering events in real time: when the user enters a segment and when the VR headset plays specific plot scenes. At the same time, a dual-queue data structure is created to store heat wave pulse commands and spray commands. After the event detection mechanism is triggered, the detection results are called and the triggering mechanism queries the mapping table to determine the corresponding haptic command type. According to the type, the command is sent to the corresponding queue: heat wave pulse commands are sent to the regular queue, and spray commands are sent to the low-latency queue.

[0029] S3. For heat wave pulse instructions in the regular queue, calculate the dynamic delay value based on the visual rendering pipeline, and drive the motion sensing worker thread to sleep according to the dynamic delay value before executing the heat wave pulse instruction.

[0030] Preferably, the formula for calculating the dynamic delay value is: In the formula, Indicates the dynamic delay value of the motion-sensing command; Indicates the baseline latency for VR visual rendering; This represents the queue load weighting coefficient; This represents the weighting coefficient based on the queue length. This indicates the number of heatwave instructions currently pending execution in the regular queue; Indicates the maximum number of instructions that a regular queue can hold; This represents the weighting coefficient for the average instruction waiting time. This represents the average waiting time for all instructions awaiting execution in the current queue; This represents the maximum allowed waiting time threshold for a single motion-sensing command in the queue. This indicates the weighting coefficient for emergency orders; This indicates the number of emergency heatwave instructions currently in the queue; This represents the CPU load weighting coefficient; This indicates the utilization rate of the GPU (Graphics Processing Unit). This represents the GPU load weighting coefficient; This indicates the utilization rate of the CPU (Central Processing Unit). This indicates the inherent response latency of motion-sensing hardware; Indicates the number of historical instruction samples; Indicates the first One historical motion-sensing command; Indicates the first The actual hardware execution timestamp of each instruction; Indicates the first The expected visual timestamp of the instruction.

[0031] Preferably, the method for driving the somatosensory working thread to sleep after a dynamic delay value and then executing the heat wave pulse command is as follows: The motion sensing worker thread acquires dynamic latency values ​​including visual rendering baseline latency, queue load prediction values, CPU and GPU load prediction values, hardware inherent latency, and historical deviation feedback values. The sleep duration is determined based on the dynamic latency value, and the user's state is set to sleep, entering a waiting state. After the hibernation period ends, the motion sensing worker thread is awakened, immediately sends an execution command to the heat wave pulse device, and records the actual hardware execution timestamp of the heat wave pulse command.

[0032] It needs to be explained that the first configuration is 200-300m. 2 A large-scale VR installation featuring 5-10 seconds of fire alarm and firefighting scenes on a flat indoor space. Hardware equipment: Heat wave equipment with an outlet air temperature of around 48℃ and water mist or small water droplet spraying equipment; VR headset, large space positioning equipment, CPU / GPU rendering host; The VR large space is divided as follows: Physical space: 10m × 10m; The fire zone is 5m x 10m on the left. The right side, 5m x 10m, is the fire extinguishing section. Motion-sensing hardware is bound to paragraphs (real hardware actions); Fire scene section → Triggered 48℃ hot air equipment to blow out heat wave; Fire extinguishing segment → Triggers water mist / small water droplet spraying equipment to spray water mist, as shown in the mapping table of segments, plot scenes and motion-triggered events; Table 1. Mapping Table of Paragraphs, Scene Scenes, and Motion-Triggered Events The binding rules are as follows: Enter the fire scene / play flames → 48℃ hot air is activated; Enter the fire zone / Play water spray → Water mist spraying starts; Participants wearing VR headsets enter from the left edge of the 10m x 10m core area to begin their experience. The specific process is as follows: As the user takes the first step into the fire scene section on the left (5m×10m), the large-space positioning device immediately collects the location information and determines that it is a fire scene section. At the same time, the VR headset switches to a fire alarm screen showing flames burning and the fire spreading (scene starts after 7 seconds). The event detection mechanism simultaneously captures two types of triggering events: entry into the section and playback of the screen. The triggering mechanism obtains the segment name fed back by the positioning device = fire segment and the judgment result = enter the fire segment. Based on this information, it searches the mapping table in Table 1 and matches the sonic command corresponding to the fire segment as the heat wave pulse command (and simultaneously matches the triggering hardware as a 48°C hot air device). The trigger mechanism determines that the current sensory command type is a heat wave pulse command. According to the original rules, the command is sent to the regular queue in the dual queue. At this time, the regular queue status is: number of commands to be executed. =1 (only the current heatwave pulse command), no emergency heatwave commands ( =0); After experiencing the fire scene for 3 seconds, the user moves to the right and enters the fire extinguishing scene (5m×10m). The positioning device determines that the scene is in the fire extinguishing section. The VR headset then switches to the scene of firefighters spraying water to extinguish the fire (scene starts after 8 seconds). The event detection mechanism captures the new trigger event. The trigger mechanism queries the mapping table and matches the instruction corresponding to the fire extinguishing section as a spraying instruction. After determining the instruction type, it is sent to the low-latency queue. The low-latency queue responds immediately without waiting in line. For heat wave pulse commands in the current conventional queue, and combining the actual performance of 48℃ hot air equipment and VR rendering host, specific values ​​are assigned to the dynamic delay formula to complete the delay calculation and drive the motion sensing device to execute synchronously. The formula parameters are set as follows; VR visual rendering benchmark latency 28ms; queue load weighting coefficient It is 2.5; It is 0.5. It is 0.3. 0.2: Queue-related weighting coefficients, summing to 1, with priority as follows: queue length > average waiting time > proportion of urgent instructions; the maximum number of instructions a regular queue can hold. The current number of heatwave instructions waiting to be executed in the queue is 15. =1 (only the 1 instruction triggered this time); current queue number of emergency heatwave instructions. =0 (No urgent instructions); Average waiting time of current queue instructions The maximum allowed latency threshold for a single instruction is 12. The time is 70ms; CPU / GPU load weighting coefficient =0.15, =0.15; GPU utilization =35% (Normal load during VR screen playback); CPU utilization =30% (background load of positioning and command scheduling); inherent response latency of motion sensing hardware The time limit is 12ms; the number of historical instruction samples is the deviation data of the 5 most recent heatwave instructions. Substitute the above parameters into the dynamic delay value of the motion control command. In the middle, dynamic delay value =28+2.1175+9.75−12−1.6=39.8675−13.6≈26.27ms; To retain one decimal place for engineering precision, the final dynamic delay value is: ≈26.3ms; S4. For the spray instructions in the low-latency queue, set a preemptive scheduling priority, so that the main control thread interrupts the heat wave pulse instruction being executed by the somatosensory worker thread and executes the spray instruction first.

[0033] Preferably, the main control thread interrupts the heat wave pulse command being executed by the motion sensing working thread and prioritizes the execution of the spray command, including: Assign a preemptive scheduling priority to the spraying instructions in the low-latency queue, and the preemptive scheduling priority is higher than the scheduling priority of all instructions in the regular queue; The main control thread monitors the low-latency queue status in real time. Once a spray command is detected to be enqueued, the execution status of the current motion-sensing worker thread is immediately obtained. Based on the execution status of the motion sensing worker thread, if it is detected that a heat wave pulse command is being executed, an interrupt signal is sent to the motion sensing worker thread and the execution of the heat wave pulse command is terminated. After the main control thread takes over the execution, it prioritizes the execution of the spray command. When the execution is completed, it sends a recovery signal to the motion sensing worker thread, allowing the motion sensing worker thread to continue processing the interrupted heat wave pulse command.

[0034] Preferably, sending an interrupt signal to the motion-sensing working thread and terminating the execution of the heat wave pulse command includes: Based on preemptive scheduling priority, the type of currently executed somatosensory instruction is dynamically compared with the priority of the enqueued spray instruction to determine whether preemption is triggered. When the spray command has a higher priority than the heat wave pulse command, the main control thread immediately interrupts the current command and generates a preemptive interrupt signal; The main control thread sends a preemption interrupt signal to the motion sensing worker thread. The motion sensing worker thread immediately suspends the current task at any point during instruction execution and saves the status information of the interrupted instruction. The main control thread confirms that the motion sensing worker thread has been suspended and switches execution control from the motion sensing worker thread to the main control thread.

[0035] It needs to be explained that the dynamic delay value is based on calculations. ≈26.3ms, the motion sensing worker thread executes the heat wave pulse command according to the following steps: The motion sensing worker thread reads the complete dynamic latency value of 26.3ms. This dynamic latency value integrates the visual rendering baseline latency, queue load, CPU / GPU load, inherent hardware latency, and historical deviation feedback, requiring no additional calculation. The thread sets its own state to sleep, with the sleep duration strictly set to 26.3ms, to ensure that the motion sensing trigger and the VR flame screen timeline are precisely aligned, avoiding the motion sensing being too early or too late. After the hibernation ends, the thread immediately wakes up and sends an execution command to the 48℃ hot air device: start the hot air, the outlet temperature is 48℃, and continue to blow until the user leaves the fire scene. After receiving the command, the hot air device blows out 48°C hot air after its inherent 12ms response delay, allowing the user to experience the heat wave in sync with the flame image. At the same time, the thread records the actual hardware execution timestamp of this command as a historical sample for the next dynamic delay calculation, which is used to optimize subsequent synchronization accuracy.

[0036] When the user enters the fire extinguishing phase, the triggered sprinkler command is sent to a low-latency queue and executed directly without calculating dynamic latency. Upon receiving the spray command, the low-latency queue immediately wakes up the motion-sensing worker thread without waiting in the queue. The thread directly sends an execution command to the water mist / small water droplet spraying device: start spraying and continue until the user leaves the fire extinguishing section; The spraying equipment starts without delay, spraying out a fine water mist / droplets, which is completely synchronized in real time with the VR water spraying fire extinguishing scene, restoring the real fire extinguishing experience. After completing the 8-second fire extinguishing scenario experience, the participant leaves the 10m×10m core area, and all motion-sensing devices automatically shut down, ending the process.

[0037] S5. Record the expected visual timestamp and the actual hardware execution time of each motion-sensing command, and use the deviation between the two as the basis for adjusting the dynamic latency value and the preemptive scheduling priority.

[0038] Preferably, the deviation between the two is used as the basis for adjusting the dynamic delay value and the preemptive scheduling priority, including: When executing each motion-sensing command, the motion-sensing worker thread records the expected visual timestamp corresponding to the command, and records the actual hardware execution time after the motion-sensing command is completed. Obtain the expected visual timestamp and the actual hardware execution time of the same instruction, and calculate the difference between the expected visual timestamp and the actual hardware execution time as the execution deviation value of the haptic instruction; The execution deviation value is stored in the deviation queue, and a preset number of the latest instruction deviation value samples are maintained according to the first-in-first-out principle; Read the deviation value samples in the deviation queue, calculate the weighted average of all deviation value samples, and use the weighted average as the basis for adjusting the dynamic delay value and the preemptive scheduling priority.

[0039] It should be explained that after the heat wave pulse command triggered when the user enters the fire scene section is executed, the motion sensing thread completes the timestamp recording and deviation calculation as required: Expected visual timestamp: The VR headset starts playing the flame burning scene at time t=1000ms. The motion sensing is set to be triggered synchronously with the scene. Therefore, the expected visual timestamp is 1000ms. Actual hardware execution timestamp: The motion sensing thread starts sleeping at t=1000ms based on a dynamic latency of 26.3ms, and wakes up at t=1026.3ms to send a motion sensing command to the 48℃ hot air device; The inherent response delay of the hot air equipment is 12ms, and the actual start time of air output is t=1026.3+12=1038.3ms; Therefore, the actual hardware execution timestamp is 1038.3ms; Execution deviation calculation: The deviation value is defined as: actual execution time - expected visual time, which results in a synchronization error of 38.3ms reflecting the somatosensory and visual perception.

[0040] A positive value indicates that the perceived motion lags behind the visual image by 38.3ms, indicating a significant asynchrony problem.

[0041] Following the first-in, first-out principle, only the deviation samples of the latest 5 motion control commands are retained; after the 38.3ms deviation value calculated in this case is added to the queue, the oldest historical deviation sample is automatically removed, and the queue is updated to contain the latest deviation data of commands 2 to 6. It should be further explained that the first-in, first-out principle means that the deviation queue is managed according to the order in which samples enter. Newly added deviation samples are placed at the end of the queue. When the queue reaches the preset capacity, the earliest entered sample is removed from the front of the queue, thereby dynamically maintaining a sample set that always contains the latest executed deviation.

[0042] All deviation samples in the queue are read, and a weighted average deviation is calculated according to the rule that the latest instruction has the highest weight and the weight of earlier instructions decreases in sequence, which focuses more on reflecting the real-time synchronization status of the current device. This weighted average deviation will be used as the core adjustment basis and will be applied to the dynamic latency value and the preemptive scheduling priority respectively. After the adjustment is completed, when the experiencer enters the fire scene next time, the haptic trigger time will be closer to the expected visual timestamp, the synchronization error between haptic and visual perception will be significantly reduced, and the smoothness and realism of the immersive experience will be effectively improved.

[0043] It should be noted that the calculation formulas and all parameters involved in the calculations in this application have been dimensionless beforehand. The process of dimensionless processing is well known in the industry and will not be described here.

[0044] Although the present application has disclosed the preferred embodiments above, the embodiments are merely examples for the purpose of illustration and are not intended to limit the present application. Those skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present application. The scope of protection claimed by the present application should be determined by the claims.

Claims

1. A method for driving a VR scene-based somatosensory device, characterized in that, The method includes: S1. In the VR headset, the VR large space is divided into fire scene segment and fire extinguishing segment, and a mapping table of plot screen and somatosensory trigger events is defined for each segment. S2. Based on the mapping table, a triggering mechanism and dual queues are constructed. The triggering mechanism determines the segment or scene that the user enters based on the mapping table and the plot screen currently being played by the VR headset, and sends the corresponding haptic commands to the regular queue or low-latency queue in the dual queues. S3. For heat wave pulse instructions in the regular queue, calculate the dynamic delay value based on the visual rendering pipeline, and drive the motion sensing worker thread to sleep according to the dynamic delay value before executing the heat wave pulse instruction. S4. For the spraying instructions in the low-latency queue, set a preemptive scheduling priority, so that the main control thread interrupts the heat wave pulse instruction being executed by the somatosensory working thread and executes the spraying instruction first. S5. Record the expected visual timestamp and the actual hardware execution time of each motion-sensing command, and use the deviation between the two as the basis for adjusting the dynamic latency value and the preemptive scheduling priority.

2. The VR scene-based motion-sensing device driving method according to claim 1, characterized in that, The mapping table defining the plot scenes and motion-triggered events for each paragraph includes: In VR headsets, the physical space within the VR large space is divided into fire scene segments and fire extinguishing segments according to the scenario and plot. Design VR scene sequences for the fire scene segment that include flames burning and heat wave radiation, and define heat wave pulse haptic trigger events based on the scene sequences of the fire scene segment; Design VR scene scenes for the fire extinguishing segment that include firefighters spraying water to extinguish fires, and define spraying motion trigger events based on the scene scenes of the fire extinguishing segment; By binding fire scene segments with heat wave pulse events and fire extinguishing segments with sprinkler events, a mapping table is formed of segments, plot scenes, and motion-triggered events.

3. The motion-sensing device driving method based on VR scenes according to claim 2, characterized in that, The step of sending the corresponding motion-sensing command into the regular queue or low-latency queue in the dual queues includes: An event detection mechanism is built based on a mapping table. This mechanism is used to monitor events in real time, such as when a user enters a segment or when a VR headset plays a specific scene. Create a dual queue consisting of a regular queue and a low-latency queue, and use the dual queue to store motion-sensing commands including heat wave pulse commands and spray commands; The detection result of the paragraph is called and the triggering mechanism queries the mapping table to determine the corresponding motion control command for the current paragraph. Based on the type of motion control command, the motion control command is sent to the corresponding queue; If it is a heat wave pulse command, it will be sent to the regular queue; If it is a spray command, it is sent to the low-latency queue.

4. The VR scene-based motion-sensing device driving method according to claim 3, characterized in that, The step of determining the motion-sensing command corresponding to the current paragraph by querying the mapping table through the triggering mechanism includes: Based on the current spatial location of the participant, determine whether the participant is in the fire scene section or the fire extinguishing section, and obtain the section name and determination result; The trigger mechanism retrieves the paragraph name and judgment result, and uses them as the basis for querying the mapping table; The mapping table is queried based on the paragraph name to match the heat wave pulse command corresponding to the fire scene paragraph or the sprinkler command corresponding to the fire extinguishing paragraph; The trigger mechanism outputs the matching result to determine whether the current paragraph corresponds to a heat wave pulse command or a spray command.

5. The VR scene-based motion-sensing device driving method according to claim 1, characterized in that, The formula for calculating the dynamic delay value is: In the formula, Indicates the dynamic delay value of the motion-sensing command; Indicates the baseline latency for VR visual rendering; This represents the queue load weighting coefficient; This represents the weighting coefficient based on the queue length. This indicates the number of heatwave instructions currently pending execution in the regular queue; Indicates the maximum number of instructions that a regular queue can hold; This represents the weighting coefficient for the average instruction waiting time. This represents the average waiting time for all instructions awaiting execution in the current queue; This represents the maximum allowed waiting time threshold for a single motion-sensing command in the queue. This indicates the weighting coefficient for emergency orders; This indicates the number of emergency heatwave instructions currently in the queue; This represents the CPU load weighting coefficient; Indicates GPU utilization; This represents the GPU load weighting coefficient; Indicates CPU utilization; This indicates the inherent response latency of motion-sensing hardware; Indicates the number of historical instruction samples; Indicates the first One historical motion-sensing command; Indicates the first The actual hardware execution timestamp of each instruction; Indicates the first The expected visual timestamp of the instruction.

6. The motion-sensing device driving method based on VR scene according to claim 5, characterized in that, The method for executing the heat wave pulse command after the driver's somatosensory working thread has gone into sleep mode according to the dynamic delay value is as follows: The motion sensing worker thread acquires dynamic latency values ​​including visual rendering baseline latency, queue load prediction values, CPU and GPU load prediction values, hardware inherent latency, and historical deviation feedback values. The sleep duration is determined based on the dynamic latency value, and the user's state is set to sleep, entering a waiting state. After the hibernation period ends, the motion sensing worker thread is awakened, immediately sends an execution command to the heat wave pulse device, and records the actual hardware execution timestamp of the heat wave pulse command.

7. The VR scene-based motion-sensing device driving method according to claim 1, characterized in that, The process of interrupting the heat wave pulse command being executed by the haptic feedback working thread and prioritizing the execution of the spray command by the main control thread includes: Assign a preemptive scheduling priority to the spraying instructions in the low-latency queue, and the preemptive scheduling priority is higher than the scheduling priority of all instructions in the regular queue; The main control thread monitors the low-latency queue status in real time. Once a spray command is detected to be enqueued, the execution status of the current motion-sensing worker thread is immediately obtained. Based on the execution status of the motion sensing worker thread, if it is detected that a heat wave pulse command is being executed, an interrupt signal is sent to the motion sensing worker thread and the execution of the heat wave pulse command is terminated. After the main control thread takes over the execution, it prioritizes the execution of the spray command. When the execution is completed, it sends a recovery signal to the motion sensing worker thread, allowing the motion sensing worker thread to continue processing the interrupted heat wave pulse command.

8. The VR scene-based motion-sensing device driving method according to claim 7, characterized in that, Sending an interrupt signal to the motion-sensing worker thread and terminating the execution of the heat wave pulse command includes: Based on preemptive scheduling priority, the type of currently executed somatosensory instruction is dynamically compared with the priority of the enqueued spray instruction to determine whether preemption is triggered. When the spray command has a higher priority than the heat wave pulse command, the main control thread immediately interrupts the current command and generates a preemptive interrupt signal; The main control thread sends a preemption interrupt signal to the motion sensing worker thread. The motion sensing worker thread immediately suspends the current task at any point during instruction execution and saves the status information of the interrupted instruction. The main control thread confirms that the motion sensing worker thread has been suspended and switches execution control from the motion sensing worker thread to the main control thread.

9. The motion-sensing device driving method based on VR scene according to claim 1, characterized in that, The method of using the deviation between the two as the basis for adjusting the dynamic delay value and the preemptive scheduling priority includes: When executing each motion-sensing command, the motion-sensing worker thread records the expected visual timestamp corresponding to the command, and records the actual hardware execution time after the motion-sensing command is completed. Obtain the expected visual timestamp and the actual hardware execution time of the same instruction, and calculate the difference between the expected visual timestamp and the actual hardware execution time as the execution deviation value of the haptic instruction; The execution deviation value is stored in the deviation queue, and a preset number of the latest instruction deviation value samples are maintained according to the first-in-first-out principle; Read the deviation value samples in the deviation queue, calculate the weighted average of all deviation value samples, and use the weighted average as the basis for adjusting the dynamic delay value and the preemptive scheduling priority.

Images