Coal mine fireproof simulation training method based on virtual reality technology

Abstract

The invention discloses a coal mine fireproof simulation training method based on a virtual reality technology, relates to the technical field of coal mine fireproof simulation training, is used for solving the problem that a virtual scene and student operation are asynchronous due to sensor data delay or misalignment in the training process, and comprises the following steps: collecting action data of a student in real time; comprise posture change, hand movement and position coordinate information. Seamless synchronization of the sensor and the virtual scene is ensured through the time synchronization algorithm and the dynamic compensation model, and the safety and effectiveness of training are improved. In combination with a sliding window, a prediction algorithm and a redundancy detection mechanism, the system has high fault-tolerant capability, and training continuity and stability are guaranteed. The dynamic path planning under the fire spreading trend is realized by adopting the A * algorithm, and the guidance is quickly adjusted and corrected when the trainee deviates from the path, so that the real-time response capability and the immersion of training are improved, and the trainee is helped to quickly make a decision and act in an emergency.

Description

Technical Field

[0001] The invention relates to the technical field of coal mine fire prevention simulation training, and in particular to a coal mine fire prevention simulation training method based on virtual reality technology. Background Art

[0002] Coal mine fire prevention simulation training based on virtual reality technology is an immersive training system built with VR technology to simulate various scenes and fire emergencies inside coal mines. In this system, trainees can intuitively experience the occurrence of fires, smoke spread and emergency escape routes through a virtual environment, while learning the use of fire extinguishers and emergency operation points. This technology improves trainees' emergency response capabilities, avoids risks in actual drills, improves the efficiency and effectiveness of training, and provides innovative solutions for coal mine safety management.

[0003] The prior art has the following deficiencies:

[0004] In coal mine fire prevention simulation training based on virtual reality technology, sensor data delay or inaccuracy is a key issue, which may cause the virtual scene to be out of sync with the trainee's operation. VR systems rely on sensors such as head trackers, handles, and positioning devices to capture trainees' movements and interactions in real time. If there is a time difference between fire dynamics (such as fire spread, escape path display) and trainee operations due to network delays, hardware failures, or algorithm processing lags, it will seriously affect judgment. For example, in an escape drill, if the speed of virtual fire spread lags behind the actual situation, trainees may choose the wrong escape route due to misjudgment, thus facing great life-threatening danger in a real fire. Therefore, ensuring high-precision synchronization of sensor data and real-time response of the system is crucial to the safety and effectiveness of training.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a coal mine fire prevention simulation training method based on virtual reality technology, through time synchronization algorithm (such as PTP or NTP) and dynamic compensation model, to achieve accurate matching of sensors and virtual scenes, to ensure seamless synchronization of trainee operations and images, to avoid misleading caused by network fluctuations or equipment failures, and to improve the safety of training. Combined with sliding window technology, prediction compensation algorithm and redundant detection mechanism, the system has high fault tolerance, maintains continuous operation in a complex network environment, and can quickly switch even if the sensor fails, improving stability and efficiency. Through the A* algorithm and time synchronization scheme, the system can dynamically plan escape routes according to the fire and trainee positions in real time, and quickly adjust the guidance, enhance immersion and emergency decision-making capabilities, so as to solve the problems in the above-mentioned background technology.

[0007] In order to achieve the above object, the present invention provides the following technical solution: a coal mine fire prevention simulation training method based on virtual reality technology, which is used to solve the problem that the virtual scene and the trainee's operation are out of sync due to sensor data delay or inaccuracy during training, comprising the following steps:

[0008] Collect students' motion data in real time, including posture changes, hand movements and position coordinate information; and remove noise and format the collected raw data through the pre-processing module;

[0009] Monitor the transmission delay of sensor data and detect abnormalities in the network transmission process; if the delay is found to exceed the set threshold, dynamically compensate the data through timestamps and prediction models to generate temporary location information to avoid data loss or scene dislocation caused by short-term network fluctuations;

[0010] The fire spread path is predicted based on the Kalman filter algorithm. The trainee's current position and movement trend are used as input parameters of the algorithm to dynamically calculate the speed and direction of the fire spread. This ensures that the fire scene and the trainee's escape route are updated synchronously in the virtual environment.

[0011] The A* path planning algorithm is used to calculate the optimal escape route in real time, and the escape route guidance is dynamically adjusted according to the trainee's current location and fire spread trend; if the trainee deviates from the planned route during the escape process, it will be corrected through voice and visual prompts;

[0012] Use the time synchronization algorithm to accurately match the sensor data with the simulation data of the virtual scene to ensure the clock consistency between different devices; eliminate the time deviation between the sensor and the system through the synchronization correction algorithm to achieve seamless synchronization between the scene and the student's operation.

[0013] Preferably, posture change refers to the rotation and tilt of the trainee's body or head in three-dimensional space; hand movement refers to the changes in the movements of the trainee's hands during training; position coordinate information refers to the changes in the trainee's position in three-dimensional virtual space, described using the X, Y, and Z axis coordinate system.

[0014] Preferably, the specific steps of monitoring the sensor data transmission delay and performing dynamic compensation are as follows:

[0015] First, the trainee's motion data is collected from multiple sensors, and a timestamp is added to each frame of data to record the exact time point when it was generated;

[0016] By monitoring the timestamp of each frame of data and the reception time when it arrives at the VR system in real time, the delay in the transmission process is calculated;

[0017] When sensor data delay or packet loss is detected, the system calls the motion compensation algorithm to make predictions based on previously collected motion data;

[0018] After the temporary data is generated, it is directly applied to the virtual scene while waiting for the lost or delayed data packet to re-arrive. Once the original data packet arrives, the system will automatically correct the position information in the scene to make the virtual environment consistent with the real operation. If the original data fails to arrive within the set time, the temporary prediction data will be deemed valid and the trainees will be notified through the sensor feedback system to ensure that they remain sensitive to scene changes.

[0019] Preferably, the steps of predicting the fire spread path based on the Kalman filter algorithm are as follows:

[0020] In the Kalman filter algorithm, the state variables and noise model of the system are first defined. The fire spread path is assumed to be a continuous motion in two-dimensional or three-dimensional space. The state vector Describe the position of the flame at the kth moment (x k ,y k ) and velocity components Secondly, you need to define the student's current location As system input, it is used to evaluate the relative distance between the flame and the student; the noise in the system is modeled as process noise and measurement noise Where Q and R are the noise covariance matrices used to describe the uncertainty of flame spread and sensor measurement, and the state transition model is: k+1 =Ax k +Bp k +w k , where A is the state transfer matrix and B is the input control matrix, which ensures that the relative positions of the fire and the trainees jointly affect the diffusion trend of the fire;

[0021] In each iteration, the prediction phase of the Kalman filter algorithm is used to calculate the next state of the flame. The prediction equation is: in, It is the predicted position and speed based on the current state, reflecting the spread trend of the fire without additional measurement data. At the same time, the system will update the covariance matrix, and the updated expression is: P k+1|k =AP k A T +Q, ensures that the system estimate includes uncertainty, taking into account the effects of process noise.

[0022] Preferably, when new measurement data arrives, the system uses the Kalman gain K k Adjust the state prediction to synchronize the virtual scene with the actual fire dynamics. The calculation formula of Kalman gain is: Kk =P k|k-1 H T (HP k|k-1 H T +R) -1 , the status after measurement update is: Among them, zk is the current flame measurement position, H is the measurement matrix, and the updated covariance is: P k =(IK k H)P k|k-1 ,Through measurement updates, the system will correct the fire spread path in real time and ensure that the ,direction and speed of the fire in the virtual scene are synchronized with the ,trainees’ movements.

[0023] Preferably, the specific steps of dynamically calculating the optimal escape route using the A* path planning algorithm are as follows:

[0024] First, a grid map model is built in the virtual mine scene. Each location in the scene is divided into several nodes. Each node represents a specific location in the mine. The path connection between each node is defined as an edge, and each edge is assigned a different cost value. During initialization, the system marks the starting point and the target point, and identifies the known obstacle area.

[0025] According to the spread trend of the fire simulation, the weight of the nodes on the edge of the fire source in the map is adjusted in real time. The weight of each node consists of two parts: G value and H value. As the fire spreads, the cost of the nodes on the edge of the fire source will increase. The system will mark the nodes on the edge of the fire source as high-risk areas or impassable areas, thus forcing the A* algorithm to avoid dangerous paths.

[0026] In the process of path planning, the search nodes are continuously expanded through the A algorithm to search the path from the starting point to the target point;

[0027] During the student's escape process, the deviation between his current position and the planned route is continuously monitored. If it is detected that the student is not moving along the guided path, the system will immediately trigger the correction mechanism, which includes voice prompts and visual feedback to guide the student back to the correct path. If the student continues to deviate, the A* algorithm will be re-called to recalculate the new escape route based on his current position and fire spread status, and update the guidance in real time.

[0028] Preferably, the specific steps of using the time synchronization algorithm to accurately match the sensor and virtual scene data are as follows:

[0029] Use the Network Time Protocol or Precision Time Protocol during the initialization phase to synchronize all sensor devices with the main system clock;

[0030] When the sensor collects the student's operation data, it adds a timestamp to each frame of data to accurately record the time when it was generated;

[0031] Use time synchronization algorithm to correct the clock error between each device and the main system;

[0032] After all data have been time-corrected, the sensor data is fused and matched with the simulation data in the virtual scene to ensure that each frame of data can be accurately applied to the virtual environment in chronological order.

[0033] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0034] The present invention uses a time synchronization algorithm (such as PTP or NTP) and a dynamic compensation model to achieve accurate matching between sensors and virtual scenes, so that each operation of the trainee is seamlessly synchronized with the virtual screen. Through high-frequency time calibration and data compensation, the system can still ensure the matching of actions and scenes, avoid virtual misleading, and improve the safety and effectiveness of training even in the case of network fluctuations or temporary equipment failure.

[0035] After combining the sliding window technology, prediction compensation algorithm and redundancy detection mechanism, the system can respond quickly and automatically correct errors when faced with data packet loss or network delay. This mechanism enables the system to have high fault tolerance, ensuring that training services can be provided continuously in complex network environments. Even if a sensor fails temporarily, the backup sensor or prediction model can intervene in a short time to avoid interruption of the training process due to equipment problems. Not only does it improve the stability of the system, it also reduces the cost of restarting after the trainee interrupts the training, improving the efficiency of the training and the learning experience of the participants.

[0036] The present invention combines the A* algorithm with a dynamic time synchronization solution to re-plan the escape route in real time based on the fire spread trend and the trainee's location. Due to the accuracy of time synchronization, the system can complete the path calculation and feedback within milliseconds, and immediately update the escape instructions in the virtual scene. When the trainee deviates from the route or the fire changes suddenly, the system quickly makes adjustments and corrects the trainee's operation through visual or voice prompts. This real-time response capability greatly enhances the immersion of training, allowing trainees to obtain a more realistic drill experience and cultivate the ability to make quick decisions and take actions in emergency situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0038] Figure 1 The present invention is a method flow chart of a coal mine fire prevention simulation training method based on virtual reality technology. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0040] The present invention provides Figure 1 A coal mine fire prevention simulation training method based on virtual reality technology is shown, which is used to solve the problem of asynchrony between the virtual scene and the trainee's operation due to sensor data delay or inaccuracy during training, and includes the following steps:

[0041] Use multiple sensors such as head trackers, handles and positioning devices to collect trainees' motion data in real time, including posture changes, hand movements and position coordinate information; the raw data collected by the sensors is denoised and formatted through a pre-processing module to improve the reliability of data transmission;

[0042] Posture changes refer to the rotation and tilt of the trainee's body or head in three-dimensional space, usually expressed in Euler angles or quaternions. Specifically, the head tracker can capture the trainee's pitch, roll, and yaw angles, reflecting the changes in the head in up and down, left and right, and rotation directions. This information is particularly critical for ensuring the synchronization of perspectives in the virtual environment. For example, when a trainee lowers his head to look at the ground or turns his head to observe the source of fire during training, the system needs to adjust the perspective of the picture in VR in real time to avoid dizziness and dislocation in the trainee. Hand movement refers to the changes in the movement of the trainee's hands during training, including gestures, grasping, releasing, and moving direction. This information is collected through a handle or hand tracking device, involving multi-degree-of-freedom movements, such as lifting, pressing down, and rotating. Hand movement data is not only used to simulate the grasping and use of fire extinguishers, but also to detect the trainee's interactive reactions in emergency situations (such as pushing away obstacles or instructing others to escape). Ensuring real-time capture and feedback of hand movements is very important for enhancing the immersion of training and the authenticity of operations. Position coordinate information refers to the change in the trainee's position in the three-dimensional virtual space, which is generally described by the X, Y, and Z axis coordinate system. The positioning device tracks the trainee's displacement in real time, such as the process of moving from one position to another. The system needs to update the fire spread status and escape route in the scene based on the position coordinates. This information is particularly critical for navigation in a simulated environment. For example, when a trainee needs to avoid flames or find an exit during an escape drill, the system must synchronize the trainee's moving position to ensure the consistency of the virtual fire scene with its motion path to avoid misleading or delayed judgment.

[0043] Monitor the transmission delay of sensor data and detect abnormalities in the network transmission process; if the delay is found to exceed the set threshold, dynamically compensate the data through timestamps and prediction models to generate temporary location information to avoid data loss or scene dislocation caused by short-term network fluctuations;

[0044] The specific steps for monitoring sensor data transmission delay and dynamically compensating for it are as follows:

[0045] The system first collects the trainee's motion data from multiple sensors (such as head trackers, handles, and positioning devices), and attaches a timestamp to each frame of data to record the exact time point when it was generated. Timestamps are usually recorded in high-precision clocks (such as nanoseconds or milliseconds) to determine the delay in data transmission during subsequent processing. This step ensures that the system can accurately track the life cycle of each data packet and provides a basis for subsequent delay calculation and compensation. For example, once a significant deviation is detected between the timestamp of sensor data and the time it is received by the system, potential delay problems can be identified.

[0046] The system calculates the delay in the transmission process by monitoring the timestamp of each frame of data and the reception time when it arrives at the VR system in real time. If the delay exceeds the set threshold (such as 50 milliseconds), the system will mark it as an abnormal delay. At the same time, the system will also evaluate the stability of network transmission through packet loss rate, jitter, and network bandwidth usage. Once abnormal fluctuations are found, such as packet loss or sudden increase in delay, the system will record this information and initiate a response mechanism. This process ensures that data transmission problems can be discovered in time to prevent delay accumulation or network instability from interfering with the training process.

[0047] When sensor data delay or packet loss is detected, the system calls a dynamic compensation algorithm to make predictions based on previously collected motion data. Typical compensation models include linear interpolation or Kalman filtering algorithms, which use the student's historical position and movement trends to calculate the current possible position information. For example, if the data packet of the handle is delayed, the system predicts the current hand position based on the student's last received data to generate temporary position information. In a fire scene, this prediction function can avoid the display of the spread of the fire being out of sync due to short-term delays, thereby ensuring that the student's judgment is not disturbed.

[0048] After generating temporary data, the system applies the data directly to the virtual scene while waiting for the lost or delayed data packet to re-arrive. Once the original data packet arrives, the system automatically corrects the position information in the scene to keep the virtual environment consistent with the real operation. If the original data fails to arrive within the set time, the system will regard the temporary prediction data as valid and notify the trainee through the sensor feedback system (such as vibration or voice prompts) to ensure that they remain sensitive to scene changes. The entire process ensures that virtual training can proceed smoothly even under network fluctuations, without causing scene misalignment or erroneous feedback due to data delays.

[0049] The fire spread path is predicted based on the Kalman filter algorithm. The trainee's current position and movement trend are used as input parameters of the algorithm to dynamically calculate the speed and direction of the fire spread. This ensures that the fire scene and the trainee's escape route are updated synchronously in the virtual environment.

[0050] The steps for predicting the fire spread path based on the Kalman filter algorithm are as follows:

[0051] In the Kalman Filter algorithm, the state variables and noise model of the system are first defined. Assuming that the fire spread path is a continuous motion in two-dimensional or three-dimensional space, the state vector Describe the position of the flame at the kth moment (x k ,y k ) and velocity components You also need to define the student's current location As the system input, it is used to evaluate the relative distance between the flame and the trainee. At the same time, the noise in the system is modeled as process noise and measurement noise Where Q and R are the noise covariance matrices used to describe the uncertainty of flame spread and sensor measurement, and the state transition model is: k+1 =Ax k +Bp k +w k , where A is the state transfer matrix and B is the input control matrix, which ensures that the relative positions of the fire and the trainees jointly affect the diffusion trend of the fire.

[0052] In each iteration, the system calculates the next state of the flame using the prediction phase of the Kalman filter algorithm. The prediction equation is: in, It is the predicted position and speed based on the current state, reflecting the spread trend of the fire without additional measurement data. At the same time, the system will update the covariance matrix, and the updated expression is: P k+1|k =AP k A T +Q, this process ensures that the system estimates contain uncertainty, taking into account the effects of process noise. For example, when the trainee moves quickly or the escape route changes, the covariance matrix will expand, indicating that the uncertainty of the prediction increases.

[0053] When new measurement data (such as the current spread of the flame) arrives, the system uses the Kalman gain K k Adjust the state prediction to synchronize the virtual scene with the actual fire dynamics. The calculation formula of Kalman gain is: K k =P k|k-1 H T (HP k|k-1 H T +R) -1 , the status after measurement update is: Among them, z k is the current flame measurement position, H is the measurement matrix. The updated covariance is: P k =(IK k H)P k|k-1 Through measurement updates, the system will correct the fire spread path in real time and ensure that the direction and speed of the fire in the virtual scene are synchronized with the trainees' movements, reducing the misalignment problem caused by delays. At the same time, this step ensures that the dynamic interaction between the fire spread and the escape route is presented in real time, providing trainees with accurate escape guidance.

[0054] The A* (A-star) path planning algorithm is used to calculate the optimal escape route in real time, and the escape route guidance is dynamically adjusted according to the trainee's current location and the fire spread trend; if the trainee deviates from the planned route during the escape process, the system will correct it through voice and visual prompts;

[0055] The specific steps of dynamically calculating the optimal escape route using the A* (A-star) path planning algorithm are as follows:

[0056] The system first builds a grid map model in the virtual mine scene, dividing each location in the scene into several nodes (Nodes), each node represents a specific location in the mine, such as corridors, exits, obstacles, etc. The path connection between each node is defined as an edge (Edges), and each edge is assigned a different cost (Cost), such as the distance or the degree of fire risk on the path. During initialization, the system marks the starting point (the student's current location) and the target point (safe exit), and identifies known obstacle areas (such as fire sources or blocked passages). This step ensures that the map model can provide a basis for the A* algorithm, enabling it to reasonably calculate feasible paths.

[0057] The system adjusts the weights of nodes in the map that are close to the fire source in real time based on the spread trend of the fire simulation. The weight of each node consists of two parts: G value (the actual cost from the starting point to the node) and H value (the estimated cost from the node to the target point, usually calculated using Euclidean distance or Manhattan distance). As the fire spreads, the cost of nodes close to the fire source will increase significantly, and the system will mark these nodes as high-risk areas or impassable areas, forcing the A* algorithm to avoid dangerous paths. This dynamic update mechanism ensures that the escape path can adapt to changes in the fire scene in real time.

[0058] During the path planning process, the system continuously expands the search nodes through the A algorithm and searches for paths from the starting point to the target point. The A* algorithm prioritizes the node with the smallest total cost (G+H) for expansion each time, and checks whether the adjacent nodes can provide a path with a lower cost. If a better path is found, the system will update the path information and add the node to the open list (Open List) to continue expanding until the target point is reached. Once an optimal path is calculated, the system will immediately display it in the virtual scene and instruct the trainee to escape along the route with visual guidance (such as arrows or path markers). This process ensures that the escape path is always the best choice in the current environment.

[0059] During the trainee's escape process, the system will continuously monitor the deviation between their current position and the planned route. If it is detected that the trainee is not following the guided path, the system will immediately trigger the correction mechanism. Correction methods include voice prompts (such as "Please return to the safe route") and visual feedback (such as flashing arrows or red marks) to guide the trainee back to the correct path. If the trainee continues to deviate, the system will re-call the A* algorithm, recalculate the new escape route based on its current position and fire spread status, and update the guidance in real time. This feedback mechanism ensures that trainees will not get lost in complex or changing environments, and minimizes the risks caused by misjudgment.

[0060] Use time synchronization algorithms to accurately match sensor data with simulation data of virtual scenes to ensure clock consistency between different devices; use synchronization correction algorithms to eliminate time deviations between sensors and systems to achieve seamless synchronization between scenes and trainees’ operations;

[0061] The specific steps for using the time synchronization algorithm to accurately match sensor and virtual scene data are as follows:

[0062] During the initialization phase, the system uses the Network Time Protocol (NTP) or Precision Time Protocol (PTP) to synchronize all sensor devices (such as head trackers, handles, and locators) with the main system clock. NTP is suitable for millisecond-level accuracy requirements, while PTP can provide microsecond-level time synchronization to meet the needs of high-precision scenarios. This step ensures that the internal clock of each device is consistent with the standard time of the central server to avoid deviations in the timestamp of the data packet due to clock drift. In addition, to prevent errors caused by network jitter, the system will be calibrated regularly to ensure that each device always maintains time synchronization.

[0063] When the sensor collects the trainee's operation data (such as position, posture, and hand movements), it adds a timestamp to each frame of data to accurately record the time when it was generated. The system pre-aligns the data packet with the main system clock based on the received timestamp. If a gap is detected between the timestamp of the device transmitting the data and the current system clock, the system will mark the data as a "delayed packet" and perform compensation processing. This process ensures that even if there is a slight delay in the data transmission process, it will not affect the synchronization of the virtual scene and the trainee's operation.

[0064] The system uses a time synchronization algorithm (such as the Berkeley algorithm or the Cristian algorithm) to correct the clock error between each device and the main system. Specifically, when the system detects that the clock deviation of a device exceeds a set threshold (such as ±2 milliseconds), it sends a synchronization request to the device and adjusts its clock to be consistent with the system time. In addition, to avoid frequent corrections affecting system performance, the system uses sliding window technology to dynamically adjust only devices that continue to have deviations. The synchronization correction process not only solves the problem of data packet timing errors, but also ensures the smoothness of data in interactions between multiple devices.

[0065] After all data has been time-corrected, the system will fuse and match the sensor data with the simulation data in the virtual scene to ensure that each frame of data can be accurately applied to the virtual environment in chronological order. For example, when a trainee turns his head, the system will immediately refresh the corresponding scene perspective to keep the virtual picture consistent with the real action. If a device data packet is temporarily lost, the system will predict and compensate based on the timestamps and motion trends of the previous frames to ensure that the picture does not freeze or misalign. The entire process ensures that every operation of the trainee can be seamlessly connected with the virtual scene, enhancing the immersion and authenticity of the training.

[0066] Implementation 1: Using PTP to achieve high-precision microsecond-level time synchronization;

[0067] In this implementation, the Precision Time Protocol (PTP) is used to achieve microsecond-level clock synchronization, ensuring that the time between each sensor device is completely consistent and meeting the needs of high-precision real-time interaction. The core of the PTP protocol is to continuously exchange time signals with the sensor slave device through a dedicated master clock device (time server), adjust the local clock of each device, and keep them on a unified time base. Unlike NTP's network-level time synchronization, PTP is particularly suitable for local area network (LAN) environments that require low latency and high precision, and can reduce time synchronization errors to the microsecond level. This is crucial for coal mine fire simulation training, because every action and fire spread change in the virtual scene requires a millisecond response to avoid trainees from perceiving any delays.

[0068] In specific applications, all sensor devices (such as head trackers, handles, and positioning devices) are connected to a time server, which broadcasts time synchronization packets to the devices and collects feedback information from the devices. PTP calculates the delay in network transmission through the round-trip transmission of timestamps, thereby eliminating the time deviation caused by network fluctuations. Whenever the sensor clock drifts, the system immediately triggers synchronization adjustment to correct its deviation to the standard time. To further improve accuracy, dedicated network cables and switches can be used between sensors and servers to reduce jitter and fluctuations in data transmission.

[0069] In addition, the solution has the ability to respond to faults quickly. When a sensor has unstable connection or time drift exceeds the allowable range, the system will switch to a backup sensor or preset prediction model in real time to avoid data synchronization jams or misalignment in the virtual scene. Since the synchronization error of PTP is at the microsecond level, the system can update the fire spread status and escape routes in a very short time, guiding trainees to make correct judgments in a timely manner, and ensuring the authenticity and immersion of training.

[0070] Implementation method 2: time synchronization strategy using dynamic correction and data compensation;

[0071] This implementation method uses the Berkeley algorithm and sliding window technology to dynamically adjust the time synchronization between sensor data and virtual scenes to ensure stable operation of the system even under complex network conditions. The Berkeley algorithm is an average time synchronization method that collects the local clock time of all sensor devices, calculates the average value, and corrects the clock of each device to the average time. This method is suitable for environments with large network fluctuations, such as wireless networks inside mines. The sliding window technology is used for the management of historical data to ensure the integrity and order of each frame of data during the correction process.

[0072] In the specific implementation, the system will trigger a global time synchronization periodically (such as every 5 seconds), collect the timestamps of all sensors, calculate the current average time, and push the result to each device. This can effectively reduce the time drift between sensors and avoid inconsistent data packets of multiple devices. If a sensor loses data packets due to network delays, the system will use sliding window technology to store the previous frames of data and predict the current position information based on trends. For example, when the handle data packet is delayed, the system calculates the possible actions of the trainee through historical records to ensure that the virtual scene is not interrupted.

[0073] This data compensation strategy is particularly suitable for dealing with emergencies during fire drills. Even when the network is unstable, the trainee's movements can still be accurately captured by the system and fed back to the virtual scene. During the escape process, if the trainee's positioning is inaccurate for a short period of time due to network fluctuations, the system can rely on the prediction model to keep the scene synchronized and avoid incorrect escape instructions. The use of sliding windows also ensures smooth data processing, allowing the system to return to normal immediately after each synchronization correction.

[0074] Implementation method 3: Fault-tolerant solution based on NTP clock synchronization and redundancy detection;

[0075] This implementation uses the Network Time Protocol (NTP) to achieve clock synchronization, and combines redundancy detection and data retransmission mechanisms to ensure high stability and fault tolerance of the system. NTP is suitable for time synchronization scenarios that require medium precision, and can control the clock errors of different devices within milliseconds. Compared with PTP, NTP has a lower implementation cost and is suitable for complex network environments. Therefore, it is particularly suitable for scenarios in coal mine simulation training where the system needs to be guaranteed to run stably for a long time.

[0076] In the specific implementation process, the system establishes a time synchronization connection with the sensor device through the NTP server. Each sensor will periodically request time updates from the NTP server and adjust its own clock according to the received time signal. To prevent network jitter from affecting synchronization accuracy, NTP uses multiple time queries and calculates the average value to offset the errors caused by some network fluctuations. In addition, to ensure the reliability of data, the system adds a timestamp and sequence number to each sensor data packet to detect the order and integrity of the data packet.

[0077] Redundancy detection mechanisms play an important role in the data synchronization process. If the system detects data packet loss or sequence errors (such as timestamp misalignment), it will immediately trigger the retransmission mechanism to request the missing data packet from the sensor. If a device does not respond for a long time, the system will enable the data stream of the backup device to ensure the continuity of the virtual scene. For example, in fire simulation training, when the trainee's escape route needs to be updated in real time, if the time of a sensor is not synchronized or the data is lost, the system will use the data of other devices to replace it to ensure that the spread of fire and path guidance are not interrupted.

[0078] This solution has high fault tolerance while ensuring time synchronization. Even in complex network environments, the application of data retransmission and redundancy mechanisms can ensure the continuity of the training process and prevent trainees from being misled by system errors. In addition, NTP's low cost and easy deployment make it very suitable for resource-constrained environments such as mines.

[0079] The present invention achieves accurate matching of sensors and virtual scenes by adopting time synchronization algorithms (such as PTP or NTP) and dynamic compensation models, so that each operation of the trainee is seamlessly synchronized with the virtual screen. This high-precision synchronization is particularly critical for escape training, because trainees need to rely on real-time feedback provided by the system to select paths and avoid risks. If the sensor data is not synchronized with the scene, the trainee may choose a dangerous route due to misjudgment. However, through high-frequency time calibration and data compensation, even in the case of network fluctuations or temporary equipment failure, the system can still ensure the matching of actions and scenes, avoid virtual misleading, and improve the safety and effectiveness of training.

[0080] After combining the sliding window technology, prediction compensation algorithm and redundancy detection mechanism, the system can respond quickly and automatically correct errors when faced with data packet loss or network delay. This mechanism enables the system to have high fault tolerance, ensuring that training services can be provided continuously in complex network environments. Even if a sensor fails temporarily, the backup sensor or prediction model can intervene in a short time to avoid interruptions in the training process due to equipment problems. This feature not only improves the stability of the system, but also reduces the cost of restarting after the trainee interrupts the training, thereby improving the efficiency of training and the learning experience of the participants.

[0081] The present invention combines the A* algorithm with a dynamic time synchronization solution, and the system can re-plan the escape route in real time according to the fire spread trend and the trainee's location. Due to the accuracy of time synchronization, the system can complete the path calculation and feedback within milliseconds, and immediately update the escape instructions in the virtual scene. When the trainee deviates from the route or the fire changes suddenly, the system quickly makes adjustments and corrects the trainee's operation through visual or voice prompts. This real-time response capability greatly enhances the immersion of training, allowing trainees to obtain a more realistic drill experience and cultivate the ability to make quick decisions and take actions in emergency situations.

[0082] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A coal mine fire prevention simulation training method based on virtual reality technology, used to solve the problem of asynchrony between virtual scenes and trainees' operations caused by sensor data delay or inaccuracy during training, characterized in that: The following steps are involved: Collect students' motion data in real time, including posture changes, hand movements and position coordinate information; and remove noise and format the collected raw data through the pre-processing module; Monitor the transmission delay of sensor data and detect abnormalities in the network transmission process; if the delay is found to exceed the set threshold, dynamically compensate the data through timestamps and prediction models to generate temporary location information to avoid data loss or scene dislocation caused by short-term network fluctuations; The fire spread path is predicted based on the Kalman filter algorithm. The trainee's current position and movement trend are used as input parameters of the algorithm to dynamically calculate the speed and direction of the fire spread. This ensures that the fire scene and the trainee's escape route are updated synchronously in the virtual environment. The A* path planning algorithm is used to calculate the optimal escape route in real time, and the escape route guidance is dynamically adjusted according to the trainee's current location and fire spread trend; If the trainee deviates from the planned route during the escape, corrections will be made through voice and visual prompts; Use the time synchronization algorithm to accurately match the sensor data with the simulation data of the virtual scene to ensure the clock consistency between different devices; eliminate the time deviation between the sensor and the system through the synchronization correction algorithm to achieve seamless synchronization between the scene and the student's operation.

2. A coal mine fire prevention simulation training method based on virtual reality technology according to claim 1, characterized in that: Posture change refers to the rotation and tilt of the student's body or head in three-dimensional space; hand movement refers to the changes in the movements of the student's hands during training; position coordinate information refers to the changes in the student's position in three-dimensional virtual space, which is described using the X, Y, and Z axis coordinate system.

3. The method for coal mine fire prevention simulation training based on virtual reality technology according to claim 1 is characterized in that: The specific steps for monitoring sensor data transmission delay and dynamically compensating for it are as follows: First, the trainee's motion data is collected from multiple sensors, and a timestamp is added to each frame of data to record the exact time point when it was generated; By monitoring the timestamp of each frame of data and the reception time when it arrives at the VR system in real time, the delay in the transmission process is calculated; When sensor data delay or packet loss is detected, the system calls the motion compensation algorithm to make predictions based on previously collected motion data; After the temporary data is generated, it is directly applied to the virtual scene while waiting for the lost or delayed data packet to re-arrive. Once the original data packet arrives, the system will automatically correct the position information in the scene to make the virtual environment consistent with the real operation. If the original data fails to arrive within the set time, the temporary prediction data will be deemed valid and the trainees will be notified through the sensor feedback system to ensure that they remain sensitive to scene changes.

4. The method for coal mine fire prevention simulation training based on virtual reality technology according to claim 1, characterized in that: The steps for predicting the fire spread path based on the Kalman filter algorithm are as follows: In the Kalman filter algorithm, the state variables and noise model of the system are first defined. The fire spread path is assumed to be a continuous motion in two-dimensional or three-dimensional space. The state vector Describe the position of the flame at the kth moment (x k ,y k ) and velocity components Secondly, you need to define the student's current location As system input, it is used to evaluate the relative distance between the flame and the student; the noise in the system is modeled as process noise and measurement noise Where Q and R are the noise covariance matrices used to describe the uncertainty of flame spread and sensor measurement, and the state transition model is: k+1 =Ax k +Bp k +w k , where A is the state transfer matrix and B is the input control matrix, which ensures that the relative positions of the fire and the trainees jointly affect the diffusion trend of the fire; In each iteration, the prediction phase of the Kalman filter algorithm is used to calculate the next state of the flame. The prediction equation is: in, It is the predicted position and speed based on the current state, reflecting the spread trend of the fire without additional measurement data. At the same time, the system will update the covariance matrix, and the updated expression is: P k+1|k =AP k A T +Q, ensures that the system estimate includes uncertainty, taking into account the effects of process noise.

5. A coal mine fire prevention simulation training method based on virtual reality technology according to claim 4, characterized in that: When new measurement data arrives, the system uses the Kalman gain K k Adjust the state prediction to synchronize the virtual scene with the actual fire dynamics. The calculation formula of Kalman gain is: K k =P k|k-1 H T (HP k|k-1 H T +R) -1 , the status after measurement update is: Among them, z k is the current flame measurement position, H is the measurement matrix, and the updated covariance is: P k =(IK k H)P k|k-1 ,Through measurement updates, the system will correct the fire spread path in real time and ensure that the ,direction and speed of the fire in the virtual scene are synchronized with the ,trainees’ movements.

6. The method for coal mine fire prevention simulation training based on virtual reality technology according to claim 1 is characterized in that: The specific steps of dynamically calculating the optimal escape route using the A* path planning algorithm are as follows: First, a grid map model is built in the virtual mine scene. Each location in the scene is divided into several nodes. Each node represents a specific location in the mine. The path connection between each node is defined as an edge, and each edge is assigned a different cost value. During initialization, the system marks the starting point and the target point, and identifies the known obstacle area. According to the spread trend of the fire simulation, the weight of the nodes on the edge of the fire source in the map is adjusted in real time. The weight of each node consists of two parts: G value and H value. As the fire spreads, the cost of the nodes on the edge of the fire source will increase. The system will mark the nodes on the edge of the fire source as high-risk areas or impassable areas, thus forcing the A* algorithm to avoid dangerous paths. In the process of path planning, the search nodes are continuously expanded through the A algorithm to search the path from the starting point to the target point; During the student's escape process, the deviation between his current position and the planned route is continuously monitored. If it is detected that the student is not moving along the guided path, the system will immediately trigger the correction mechanism, which includes voice prompts and visual feedback to guide the student back to the correct path. If the student continues to deviate, the A* algorithm will be re-called to recalculate the new escape route based on his current position and fire spread status, and update the guidance in real time.

7. The method for coal mine fire prevention simulation training based on virtual reality technology according to claim 1 is characterized in that: The specific steps for using the time synchronization algorithm to accurately match sensor and virtual scene data are as follows: During the initialization phase, the Network Time Protocol or Precision Time Protocol is used to synchronize all sensor devices with the main system clock; When the sensor collects the student's operation data, it adds a timestamp to each frame of data to accurately record the time when it was generated; Use time synchronization algorithm to correct the clock error between each device and the main system; After all data have been time-corrected, the sensor data is fused and matched with the simulation data in the virtual scene to ensure that each frame of data can be accurately applied to the virtual environment in chronological order.

Images