Fire extinguishing teaching system based on motion capture

Abstract

The present application relates to the field of virtual reality simulation and human-computer interaction, and particularly relates to a fire extinguishing teaching system based on motion capture, comprising: a data acquisition module, configured to acquire real-time motion capture data stream and state data stream; a reference reconstruction module, configured to solve an ideal sweep trajectory covering the root of a virtual fire source as reference data; a fault simulation module, configured to generate a labeled theoretical deviation feature library and corresponding multiple sets of parallel simulation trajectories; a difference extraction module, configured to generate a real residual vector and a theoretical residual vector set; a topological coupling module, configured to map the real residual vector to the feature space of the theoretical residual vector set to calculate a topological similarity, and select the deviation type corresponding to the highest similarity as a user intention recognition result; and a feedback generation module, configured to generate a feedback instruction according to the user intention recognition result. The present application significantly reduces the initialization data dependency and dependence on real samples of the system.

Description

Technical Field

[0001] This invention relates to the field of virtual reality simulation and human-computer interaction technology, specifically to a fire extinguishing teaching system based on motion capture. Background Technology

[0002] Current virtual reality firefighting skills training primarily utilizes inertial measurement units (IMUs) and optical positioning technology to drive virtual model drills. To determine operational compliance, existing solutions generally employ static standard trajectory matching or supervised learning classification techniques based on massive amounts of real samples, evaluating results by calculating geometric deviations. However, this passive approach has significant limitations: firstly, it heavily relies on the collection and labeling of large-scale real error data, resulting in high dependence on initial system data and weak generalization ability for rare error patterns; secondly, simple Euclidean space error calculation cannot deeply decouple specific operational intentions, failing to effectively distinguish between directional errors caused by technical cognitive biases and random jitter caused by physiological limitations. This often limits teaching feedback to simple success or failure assessments, lacking dynamic analysis of error causes and targeted correction capabilities.

[0003] Therefore, how to achieve reverse computation of user operation intentions and real-time accurate classification of error patterns without relying on massive amounts of prior data, so as to provide adaptive feedback that conforms to physical constraints, has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a fire extinguishing teaching system based on motion capture, and to solve the following technical problems: It avoids the non-adaptive limitations of fixed-script teaching and the dependence on training with massive amounts of real error data. It can also reverse-engineer user operation intentions through active fault simulation and intelligently distinguish between technical errors and physiological limitations.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: The data acquisition module is used to acquire real-time motion capture data streams containing spatial position coordinates, attitude quaternions and acceleration components of the handheld fire extinguisher simulator, as well as status data streams containing the dynamic coordinates of the virtual fire source and fire intensity parameters. The baseline reconstruction module is used to construct an ideal interactive reference space based on the state data stream and solve for the ideal sweeping trajectory covering the root of the virtual fire source as baseline data. The fault simulation module is used to parameterize preset error modes and inject them into the ideal interactive reference space to generate a labeled theoretical deviation feature library and corresponding multiple sets of parallel simulation trajectories. The difference extraction module is used to calculate the difference between the real-time motion capture data stream and the ideal interactive reference space to generate a real residual vector, and to calculate the difference between each set of parallel simulation trajectories and the ideal interactive reference space to generate a theoretical residual vector set. The topology coupling module is used to map the real residual vector to the feature space of the theoretical residual vector set, calculate the topological similarity between the mapped real residual vector and each vector in the theoretical residual vector set, and select the deviation type corresponding to the highest similarity as the user intent recognition result. The feedback generation module is used to generate feedback instructions based on the user intent recognition results.

[0006] As a further aspect of the present invention: the fault simulation module includes: The geometric deviation injection unit is used to simulate the pointing input deviation. It superimposes a preset elevation deviation vector or azimuth deviation vector on the spatial vector of the ideal sweeping trajectory to generate a parallel simulation trajectory that simulates the aiming offset. The timing warp unit is used to simulate rhythmic input deviations by compressing or stretching the time axis of the ideal sweeping trajectory to generate a parallel simulation trajectory that simulates abnormal sweeping speed. The noise superposition unit is used to simulate physiological or unstable inputs by superimposing random frequency noise that conforms to muscle fatigue characteristics or specific waveforms that conform to sensor drift characteristics onto the ideal sweeping trajectory to generate a parallel simulation trajectory that simulates hand tremor.

[0007] As a further aspect of the present invention: the difference extraction module includes: The synchronization alignment unit is used to perform spatiotemporal alignment of the data before calculating the difference. It uses a dynamic time warping algorithm to map the time series of the real-time motion capture data stream onto the time axis of the ideal interactive reference space. The pure feature extraction unit is used to eliminate reference signal interference. Through vector subtraction, the reference data of the ideal sweeping trajectory is subtracted from the parallel simulation trajectory to obtain a theoretical residual vector set containing only the decoupled input deviation features.

[0008] As a further aspect of the present invention: the topology coupling module includes: The projection computation unit is used to perform dimensionality reduction projection of the feature space, projecting the real residual vector and the theoretical residual vector set onto a unified metric space; The similarity decision unit is used to calculate the cosine similarity or Euclidean distance between the projected real residual vector and each theoretical residual vector, and to construct the similarity scoring matrix. The state locking unit is used to lock the dominant input bias pattern based on the similarity score matrix. When the similarity score between the actual residual vector and the theoretical residual vector of a certain type is higher than or equal to the preset confidence threshold, the specific type is determined as the current dominant input bias pattern.

[0009] As a further aspect of the present invention: the state locking unit is also configured with noise reduction decision logic: When the similarity score between the actual residual vector and the theoretical residual vector of the simulated aiming offset is higher than or equal to the confidence threshold, and the similarity score between the actual residual vector and the theoretical residual vector of the simulated handshake is lower than or equal to the confidence threshold, the user intent recognition result is determined to be a target pointing error. When the similarity score between the actual residual vector and the theoretical residual vector of the simulated aiming offset is lower than or equal to the confidence threshold, and the similarity score between the actual residual vector and the theoretical residual vector of the simulated handshake is higher than or equal to the confidence threshold, the user intent recognition result is determined to be physiological shaking or insufficient control, and a non-corrective interactive feedback prompting stable operation is generated. When the similarity scores of the actual residual vector, the theoretical residual vector of the simulated aiming offset, and the theoretical residual vector of the simulated handshake are all higher than or equal to the confidence threshold, it is judged as a composite error. Corrective feedback for the target pointing error is output first, and a stable operation prompt is superimposed at the same time. When the similarity scores of the actual residual vector, the theoretical residual vector of the simulated aiming offset, and the theoretical residual vector of the simulated handshake are all lower than or equal to the confidence threshold, it is determined to be an atypical operation or an error mode not defined by the system, and no correction instruction is generated.

[0010] As a further aspect of the present invention: the benchmark reconstruction module includes: The physical constraint unit is used to define the motion boundary of virtual object interaction, and sets the activity limit range and maximum movement speed of the virtual representative point according to human kinematic parameters. The optimal path planning unit is used to calculate the most efficient sweeping path. Based on the dynamic coordinates of the virtual fire source and the current virtual wind direction parameters, the path planning algorithm is used to solve the ideal sweeping trajectory that minimizes the time required for the virtual extinguishing agent to cover the root of the fire source within the movement boundary.

[0011] As a further aspect of the present invention: the feedback generation module includes: The visual enhancement unit is used to overlay and display visual guidance in virtual reality display devices, overlaying the ideal scanning trajectory as a semi-transparent path in the user's field of vision, and visualizing the real residual vector as an operation deviation indicator; The haptic feedback unit drives the vibration motor of the handheld device. In response to the user's intention recognition result as an input deviation, the vibration intensity of the vibration motor is modulated according to the modulus of the actual residual vector.

[0012] As a further aspect of the present invention, the data acquisition module further includes: The preprocessing unit is used to filter and preprocess the real-time motion capture data stream. It uses the Kalman filter algorithm to smooth the spatial position coordinates and attitude quaternions, and removes outlier data caused by sensor occlusion.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention injects preset error modes into an ideal interactive reference space through a fault simulation module to generate labeled parallel simulation trajectories. This approach establishes a complete theoretical deviation feature library, which can cover the entire spectrum of teaching scenarios, including rare operational errors, without relying on massive amounts of real error data for training. This significantly reduces the system's dependence on initialization data and real samples. 2. Through the denoising decision logic of the topology coupling module and the state locking unit, the system can map the real residual vector to the theoretical feature space to perform topological similarity calculation. This enables the system to effectively distinguish between orientation errors caused by cognitive biases and random jitters caused by physiological limitations, thereby avoiding misjudgments caused by unintentional physiological tremors of users and enhancing the pertinence and intelligence of teaching feedback. 3. The benchmark reconstruction module of this invention utilizes physical constraint units and optimal path planning units, combined with the dynamic coordinates of the virtual fire source, wind direction parameters, and human kinematic parameters, to calculate in real time the ideal sweeping trajectory that conforms to the current environment and human function. This ensures that the teaching benchmark can be dynamically adjusted with changes in the virtual scene, avoiding the non-adaptive limitation problem of traditional fixed script teaching, and always providing the best operating template that conforms to physical laws. 4. The differential extraction module of this invention integrates a synchronization alignment unit and a clean feature extraction unit. It uses a dynamic time warping algorithm to solve the problem of spatiotemporal asynchrony and removes the interference of the reference signal through vector subtraction. With the Kalman filtering algorithm of the preprocessing unit in the data acquisition module, the system can obtain a residual vector containing only the decoupled input deviation features, which eliminates the interference of time phase difference and sensor noise on intention recognition and greatly improves the accuracy of user operation status determination. 5. The feedback generation module of this invention combines a visual enhancement unit and a tactile feedback unit. It not only overlays the ideal path and deviation markers in the virtual field of vision, but also dynamically modulates the vibration intensity of the handheld device according to the magnitude of the residual vector. This feedback method, which provides visual guidance by the visual enhancement unit and vibration stimulation by the tactile feedback unit, not only intuitively displays the direction and degree of operational errors, but also strengthens muscle memory through tactile stimulation, helping users to quickly correct their actions in complex dynamic interactions, and significantly improving the training effect of firefighting skills. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the structure of a fire extinguishing teaching system based on motion capture provided in an embodiment of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] This invention proposes a fire extinguishing teaching system based on motion capture. Please refer to [link / reference]. Figure 1 The system includes: a data acquisition module, used to acquire real-time motion capture data streams containing spatial position coordinates, attitude quaternions and acceleration components of the handheld fire extinguisher simulator, and a status data stream containing the dynamic coordinates of the virtual fire source and fire intensity parameters; The baseline reconstruction module is used to construct an ideal interactive reference space based on the state data stream and solve for the ideal sweeping trajectory covering the root of the virtual fire source as baseline data. The fault simulation module is used to parameterize preset error modes and inject them into the ideal interactive reference space to generate a labeled theoretical deviation feature library and corresponding multiple sets of parallel simulation trajectories. The difference extraction module is used to calculate the difference between the real-time motion capture data stream and the ideal interactive reference space to generate a real residual vector, and to calculate the difference between each set of parallel simulation trajectories and the ideal interactive reference space to generate a theoretical residual vector set. The topology coupling module is used to map the real residual vector to the feature space of the theoretical residual vector set, calculate the topological similarity between the mapped real residual vector and each vector in the theoretical residual vector set, and select the deviation type corresponding to the highest similarity as the user intent recognition result. The feedback generation module is used to generate feedback instructions based on the user intent recognition results.

[0018] This embodiment details the core architecture of the system, which employs a synthetic analysis architecture. By constructing an ideal benchmark and actively simulating fault modes, it achieves inverse computation of user operation intentions. The data acquisition module uses a combination of a high-frequency inertial measurement unit and optical positioning markers to acquire real-time motion capture data streams at a frequency of 120Hz. The data stream is defined as follows: in, It originates from spatial positioning calculations and its physical meaning is a spatial position coordinate vector; This represents the total duration of data collection. Derived from the attitude sensor, its physical meaning is the attitude quaternion representing the direction of the fire extinguisher nozzle; Derived from accelerometers, its physical meaning is the acceleration component used for jitter analysis; Construct symbols for sets; This is a condition separator; Mathematical symbols belong to; : Symbol for a closed interval in mathematics Indicates a time range from From the moment on, until The time has ended, and includes and Two endpoints.

[0019] At the same time, the system extracts the coordinates of the root center of the virtual fire source from the rendering state of the physics engine. and fire intensity scalar State data flow; the benchmark reconstruction module constructs an ideal interactive reference space. The manifold is defined as a three-dimensional special Euclidean subspace that satisfies the geometric constraints of effective fire suppression coverage, specifically defined as follows: in, It is a spatial position coordinate vector; For attitude quaternions; Represents the three-dimensional real Euclidean space; Describe the Lie group consisting of unit quaternions; Represents the 2-norm of a vector; operators This is vector subtraction, used here to calculate the current point. Relative to shoulder joint reference point The displacement vector; Here is the attitude rotation matrix. The unit vector along the nozzle axis, superscript The transpose operation represents a matrix or vector. For cosine trigonometric functions, The preset effective spray angle, typically valued at... This value is based on the experimental determination of the fluid scattering cone angle of a standard dry powder fire extinguisher nozzle within its effective range; Under this manifold constraint, the system can be programmed to give the energy functional as: in, Position vector The first derivative with respect to time represents the linear velocity vector; the minimum geodesic is taken as the ideal sweep trajectory. The norm of the quaternion derivative term is defined as: in, For attitude quaternions The first derivative with respect to time; Let be the angular velocity vector, with dimensions . To ensure consistency of physical dimensions, the weighting coefficients are configured as follows: and ; here The physical meaning is the square of the equivalent radius of gyration. ,in Used to map angular velocity energy to equivalent linear velocity energy , Representing the fundamental physical dimensions of length; thus, related to the translation term. Dimensions are Weighted summation is performed under the same physical dimensions; The coefficient value is obtained by solving the inverse optimal control of the expert-taught trajectory, aiming to optimize the relative proportion of balance path smoothness and attitude stability in the target. The fault simulation module, as the core generator, does not passively identify faults but actively generates hypotheses by calling the operation error injection engine. This module injects preset error parameters... Generate multiple sets of parallel simulation trajectories ,in, Indicates the first Index of preset error modes , The total number of preset error patterns; each track represents a specific, labeled error pattern; Based on this, the difference extraction module performs dual-track calculations to obtain the actual residual vectors containing user mixing errors. and the theoretical residual vector set containing only pure error features The topology coupling module calculates... and Topological similarity of elements The best match is found in the feature space; the similarity calculation uses the cosine metric in the feature space, and its formula is as follows: in, For the first The eigenvectors of the theoretical residual vector set after processing by the projection computation unit; This allows for the quantification of the alignment between actual operational deviations and hypothetical failure modes in terms of geometric structure, thereby enabling the determination of the user's current error type.

[0020] As a preferred embodiment of the present invention, the reference reconstruction module includes: a physical constraint unit, used to define the motion boundary of virtual object interaction, and to set the activity limit range and maximum movement speed of the virtual representative point according to human kinematic parameters; The optimal path planning unit is used to calculate the most efficient sweeping path. Based on the dynamic coordinates of the virtual fire source and the current virtual wind direction parameters, the path planning algorithm is used to solve the ideal sweeping trajectory that minimizes the time required for the virtual extinguishing agent to cover the root of the fire source within the movement boundary.

[0021] Here, interaction efficiency refers to a comprehensive quantitative evaluation of the accuracy of spatial coverage and the smoothness of action execution during the firefighting mission. Specifically, the highest interaction efficiency means that, under the premise of satisfying the physical motion boundary constraints, the trajectory planned by the system can achieve full coverage of the root of the dynamic fire source with minimal path redundancy and the most stable dynamic characteristics; it is reflected in the subsequent cost function. The minimum value is achieved by ensuring that the virtual extinguishing agent is always aimed at the root of the fire source with high precision, while ensuring the continuity and stability of the sweeping action, thereby avoiding ineffective reciprocating motion or violent shaking, and achieving the dual optimization goal of maximizing the utilization rate of the extinguishing agent and minimizing the operation time.

[0022] In this embodiment, the physical constraint unit defines the motion boundary of the virtual representative point, and its calculation formula is as follows: in, This represents the set of valid spatial regions where activities are permitted for virtual representative points; Represents the position coordinate vector of any point in three-dimensional space; shoulder joint reference coordinates Based on real-time coordinates of the head-mounted display Superimposed preset ergonomic offset vector The calculation shows that, , Typical value Arm limit ; The specific human kinematic parameters here include the shoulder joint reference coordinates, which serve as the benchmark for movement. Preset ergonomic offset vector reflecting human body structure And the arm limit determined by the length of the human arm. It should be noted that the activity limit range does not refer to a geographical address range, but rather to the user's current shoulder joint reference coordinates. For the center of the ball, the extreme limit of the arm The local spherical space region constructed with radius represents the effective human-computer interaction workspace that a user can reach while holding a fire extinguisher in a standing position; Maximum movement speed The relationship between the motion boundary and the virtual representative point is one of complementary constraints: the motion boundary defines the spatial geometric boundary that the virtual representative point can reach, while the maximum moving speed defines the upper limit of the dynamics of the virtual representative point moving within this space; when generating the trajectory, the system must simultaneously ensure that the trajectory point is located within the motion boundary and that the rate of change of displacement of adjacent trajectory points does not exceed the maximum moving speed, thereby ensuring that the teaching demonstration movements neither exceed the range of the human arm length nor violate the rate limit of normal human muscle drive; the optimal path planning unit will generate the ideal trajectory Discretize into Step sequence ; Introducing wind direction correction vector ,in, The wind speed vector in the virtual environment, in units of , Here are the drift coefficients; the cost function is constructed as follows: in, For summation index at discrete time steps; For the first The coordinates of the virtual fire source root center corresponding to each discrete time step; For the first Spatial position coordinates on the ideal sweep trajectory at each discrete time step; The spatial coordinates of the ideal sweep trajectory at the previous time step; The time step for discretization is set to a value of [value]. In this formula, The physical meaning is the discrete linear velocity vector of the virtual representative point between adjacent time steps, calculated by the first-order forward difference approximation. This term is related to the smoothing weight. This design aims to penalize excessive speed abrupt changes in the trajectory to ensure the continuity and stability of the strafing action; Location tracking weights Preferred range is The physical meaning of this parameter corresponds to the square of the natural frequency of the second-order system. This is used to define the equivalent stiffness to mass ratio of the virtual guiding force field to ensure that the dynamic response characteristics of trajectory tracking meet the critical damping requirements; smoothing weights Preferred range is The constraint is set as follows: maximum speed. , upper limit of acceleration ; The system calls the sequence quadratic programming solver to calculate the optimal control point sequence. The solution process uses a linear interpolation path from the starting point to the target point as the initial value. The Hessian matrix is ​​updated approximately using a quasi-Newton method to ensure convergence stability under non-convex constraints. This embodiment ensures the dynamic adaptability of teaching standards by reconstructing the benchmark in real time based on physical constraints and optimal control theory. No matter how the virtual fire source moves or how the wind direction changes, the system can always provide an optimal operating template that conforms to the current environment and human function, avoiding the non-adaptive limitation problem of fixed script teaching and enhancing the realism of the teaching scenario.

[0023] As a preferred embodiment of the present invention, the fault simulation module includes: a geometric deviation injection unit, used to simulate the pointing input deviation, superimposing a preset elevation angle deviation vector or azimuth angle deviation vector on the spatial vector of the ideal sweeping trajectory to generate a parallel simulation trajectory simulating aiming offset; The timing warp unit is used to simulate rhythmic input deviations by compressing or stretching the time axis of the ideal sweeping trajectory to generate a parallel simulation trajectory that simulates abnormal sweeping speed. The noise superposition unit is used to simulate physiological or unstable inputs by superimposing random frequency noise that conforms to muscle fatigue characteristics or specific waveforms that conform to sensor drift characteristics onto the ideal sweeping trajectory to generate a parallel simulation trajectory that simulates hand tremor.

[0024] This embodiment further defines the parameterized generation logic of the fault simulation module; it generates attitude deviations, where the directional input deviation refers to the systematic spatial angle error generated when the user establishes the aiming line of sight. The spatial vector refers to the unit vector representing the direction of extension of the fire extinguisher nozzle axis in the three-dimensional Cartesian coordinate system. The elevation angle deviation vector or azimuth angle deviation vector refers to the constant angular offset components superimposed on the vertical elevation plane and the horizontal azimuth plane, respectively. The geometric deviation injection unit is defined by the following formula: in, For the ideal strafing trajectory The ideal quaternion at the current moment; This is the quaternion multiplication operator; Indicates about the axis Rotation angle The formula for calculating a unit quaternion is: in, For angle Radius representation: Value ; They are unit vectors The X, Y, and Z axis components; Deviation axis A random unit vector perpendicular to the line of sight, whose attitude matrix is ​​based on the ideal trajectory at the current moment. The formula for generating uniform sampling is as follows: in, For ideal posture quaternions The resulting attitude rotation matrix; In order to be in The uniformly distributed random sampling phase angle within the interval is used to characterize the rotational randomness of the deviation axis in the normal plane perpendicular to the nozzle axis. and They are trigonometric functions of cosine and sine, respectively, through Construct the normal vector in the nozzle coordinate system to ensure that the injected geometric deviation is always orthogonal to the current injection direction; superscript The transpose operation represents a matrix or vector, used to convert a row vector into a column vector to meet the dimensional requirements of matrix multiplication. Through this mapping mechanism, the system couples random radial disturbances in the local coordinate system to the global spatial coordinate system, thereby ensuring the geometric consistency of the deviation relative to the nozzle coordinate system. For rhythmic input deviations—that is, the nonlinear stretching of the time axis caused by the deviation between the user's actual sweeping action execution rate and the ideal dynamic time series calculated by the system based on human dynamics constraints and optimal fire extinguishing efficiency—the time-series distortion unit defines a remapping function: Wherein, scaling factor For physiological or unstable inputs, i.e., functional tremors or hardware signal drift not caused by the user's subjective intent, random frequency noise refers to a broadband random oscillation signal simulating the high-frequency fatigue tremor characteristics of muscles, and specific waveforms refer to deterministic trend signals simulating the zero-point drift of sensors or the low-frequency gravitational traction characteristics. The noise superposition unit uses a spectral synthesis method to generate a white noise sequence. And introduce in the frequency domain Uniformly distributed random phase To eliminate signal periodicity, amplitude modulation is used. After completing the spectral modulation, an inverse Fourier transform is performed to obtain the colored noise. Where k is the discrete frequency index. , To prevent small positive numbers from diverging at zero frequency, or to set The kinematic equations of composition are: The formula for calculating the location is: The attitude calculation formula is: in, Represents the ideal strafing trajectory In the spatial location vector after time-warped mapping, It is a remapping function defined by the timing warp unit, used to simulate abnormal sweeping speed; The positional colored noise is a random signal generated by spectral synthesis that conforms to the characteristics of physiological tremor and is used to simulate the displacement of hand tremor. The gesture-colored noise is a normalized three-dimensional noise vector used to simulate the instability of handheld devices in the pointing direction. The trajectory quaternion with geometric deviation output by the aforementioned geometric deviation injection unit; function Represents the Lie algebra to unit quaternion group The exponential mapping is used to convert rotation vectors into quaternions; The diagonal gain matrix is ​​specifically represented as follows: ; scalar coefficients Used to normalize three-dimensional noise vectors It is mapped to a perturbation in the Lie algebra space; This embodiment establishes a complete error feature dictionary by parametrically injecting multiple error modes. This allows the system to cover the entire spectrum of teaching scenarios, including rare operational errors, without relying on massive amounts of real error data for training. This significantly reduces the system's dependence on initialization data and the need for real samples.

[0025] As a preferred embodiment of the present invention, the data acquisition module further includes: a preprocessing unit, used to perform filtering preprocessing on the real-time motion capture data stream, using the Kalman filter algorithm to smooth the spatial position coordinates and attitude quaternions, and to remove outlier data caused by sensor occlusion; The difference extraction module includes: a synchronization alignment unit, which performs spatiotemporal alignment of the data before calculating the difference, and uses a dynamic time warping algorithm to map the time series of the real-time motion capture data stream onto the time axis of the ideal interactive reference space; The pure feature extraction unit is used to eliminate reference signal interference. Through vector subtraction, the reference data of the ideal sweeping trajectory is subtracted from the parallel simulation trajectory to obtain a theoretical residual vector set containing only the decoupled input deviation features.

[0026] In this embodiment, the preprocessing unit constructs a discrete Kalman filter for a constant velocity motion model; the state vector is defined, and its calculation formula is as follows: in, Representing discrete time The state estimation vector is the core state representation of the Kalman filter used to smooth motion trajectories. For location, These represent the coordinate components of the position vector along the X, Y, and Z axes in a three-dimensional Cartesian coordinate system, respectively. For speed, These represent the velocity components of the velocity vector along the X, Y, and Z axes, respectively. This indicates that the state vector belongs to a six-dimensional real Euclidean space; the state transition matrix is ​​set as follows: in, express identity matrix express Zero matrix; sampling time The observation matrix is ​​set as follows: Directly mapping position coordinates; process noise covariance matrix Measure the noise covariance matrix Outlier detection calculates the Mahalanobis distance using the following formula: in, For a moment The prior state estimation vector, Let be the prior observation estimation vector, representing the predicted state vector through the observation matrix. The estimated value obtained after mapping to the observation space; For a moment The measured position observation vector, To create the covariance matrix, its calculation formula is as follows: in, Let be the prior estimate error covariance matrix; if Then, the update is refused; the synchronization alignment unit performs dynamic time warping to construct the cost matrix. The matrix is ​​based on the recursive formula: in, For the first in the real-time motion capture data stream One time series element; For the ideal interactive reference space (baseline data) in the first One time axis mapping point; The mathematical minimum sign is used to find the optimal alignment path in the current step during recursive computation; to solve the quaternion double cover problem, a modified metric function is adopted: in, , These represent the spatial coordinate vectors of the real-time motion data and the baseline data at the corresponding time. These represent the attitude quaternions of real-time motion data and baseline data at the corresponding moment, respectively, and are used to characterize the direction of the fire extinguisher nozzle; This represents the dot product operation of vectors or quaternions. Indicates taking the absolute value; The weight coefficients are those defined above; the algorithm finds the mapping relationship that minimizes the normalized path distance. ,in, Represents the first in the real-time motion capture data sequence A time index, This represents the time index in the corresponding ideal reference sequence; the pure feature extraction unit is based on the mapping path. Perform heterogeneous difference operations: For position and acceleration data in Euclidean space, the standard vector subtraction is performed, and the calculation formula is as follows: in, This represents the spatial position residual vector between the simulated trajectory and the ideal trajectory at the aligned time point; This represents the time point of the parallel simulation trajectory after dynamic time warping mapping. Spatial location data; This indicates the ideal strafing trajectory at the corresponding mapped time point. Spatial location data; for quaternion attitude data of Riemannian manifolds, logarithmic mapping difference is performed, and its calculation formula is: in, This represents the quaternion attitude residual vector between the simulated trajectory and the ideal trajectory at the aligned time point; This indicates the parallel simulation trajectory at the mapped time point. The quaternion of the position; This indicates the ideal strafing trajectory at the mapped time point. The quaternion of the position; It is a quaternion logarithmic mapping with the base being the natural constant. ,like ,when hour, , The inverse cosine function is used to extract the rotation angle scalar from a unit quaternion; when When the rotation angle is 0, it is conventional to... Coefficients in the formula This is used to transform the logarithmic mapping result into the corresponding three-dimensional rotation vector; the final combined output is the theoretical residual vector set. Its elements are concatenated vectors ;symbol This represents quaternion multiplication. As a preferred embodiment of the present invention, the topological coupling module includes: a projection calculation unit, used to perform dimensionality reduction projection of the feature space, projecting the real residual vector and the theoretical residual vector set onto a unified metric space; The similarity decision unit is used to calculate the cosine similarity or Euclidean distance between the projected real residual vector and each theoretical residual vector, and to construct the similarity scoring matrix. The state locking unit is used to lock the dominant input bias pattern based on the similarity score matrix. When the similarity score between the actual residual vector and the theoretical residual vector of a certain type is higher than or equal to the preset confidence threshold, the specific type is determined as the current dominant input bias pattern. The state locking unit is also equipped with denoising decision logic: when the similarity score between the real residual vector and the theoretical residual vector of the simulated aiming offset is higher than or equal to the confidence threshold, and the similarity score between the real residual vector and the theoretical residual vector of the simulated handshake is lower than or equal to the confidence threshold, the user intent recognition result is determined to be target pointing error. When the similarity score between the actual residual vector and the theoretical residual vector of the simulated aiming offset is lower than or equal to the confidence threshold, and the similarity score between the actual residual vector and the theoretical residual vector of the simulated handshake is higher than or equal to the confidence threshold, the user intent recognition result is determined to be physiological shaking or insufficient control, and non-corrective interactive feedback prompting stable operation is generated. When the similarity scores of the actual residual vector, the theoretical residual vector of the simulated aiming offset, and the theoretical residual vector of the simulated handshake are all higher than or equal to the confidence threshold, it is judged as a compound error. Corrective feedback for the target pointing error is output first, and stable operation prompts are superimposed at the same time. When the similarity scores of the actual residual vector, the theoretical residual vector of the simulated aiming offset, and the theoretical residual vector of the simulated handshake are all lower than or equal to the confidence threshold, it is judged as an atypical operation or an error mode not defined by the system, and no correction instruction is generated.

[0027] In this embodiment, the projection calculation unit performs resampling on the residual sequence after dynamic time warping and alignment, wherein... The position residual vector and attitude residual vector at each time step are concatenated. The 6-dimensional residual vectors of each frame are then stitched together in time step order to generate the dimensional... The original feature vector; Perform physical dimension alignment by multiplying the rotation components in the vector by the characteristic length coefficient. The value of this coefficient is derived from the statistical value of the average turning radius of an adult male forearm plus a handheld fire extinguisher simulator, and is used to convert the angle difference into an equivalent arc length difference. To construct a unified manifold, the system collects currently generated... Constructing a sample matrix from group theory residual vectors Calculate the sample mean vector Dimension The centered covariance matrix is ​​calculated as follows: in, It is a vector copy A matrix composed of columns; for Perform singular value decomposition and select the cumulative variance contribution rate. The former The eigenvectors constitute the projection basis matrix. In this embodiment, for the original 600-dimensional feature vector, experimental measurements show that... The typical value range is 20 to 50, at which point more than 95% of the information can be retained; the actual residual vector Projected as: The similarity decision unit calculates the cosine similarity, and the calculation formula is as follows: Specifically, the system will process all parallel simulation trajectories corresponding to... Similarity is calculated for each group of theoretical residual vectors. Iterate through the entire similarity set The maximum similarity value is located through comparison operations. and its corresponding index That is, satisfying The system then retrieves the data from the first... The theoretical residual vector is pre-bound with a fault mode label, and this label is directly used as the initial result of user intent recognition at the current moment. This process is essentially to find the theoretical fault model in the feature space that is closest to the geometric features of actual operation, thereby realizing the reverse mapping from data features to intent semantics.

[0028] The state-locking unit uses a pre-built offline calibration dataset to determine the decision threshold. This dataset contains positive and negative sample pairs, consisting of parallel simulated trajectories and ideal trajectories generated by injecting preset error patterns. Based on this dataset, the receiver's operational characteristic curve is calculated, and an equal error rate balance point where the false recognition rate and the rejection rate are equal is selected. The similarity threshold is determined accordingly. , used to execute the noise reduction decision logic; This embodiment achieves intelligent differentiation between technical errors and physiological limitations through topological coupling and denoising decision logic; this avoids the system incorrectly prompting the user to aim at the fire source when the user's hand is shaking due to nervousness, thereby eliminating invalid feedback and enhancing the intelligence level of the teaching system.

[0029] As a preferred embodiment of the present invention, the feedback generation module includes: a visual enhancement unit, used to overlay and display visual guidance in a virtual reality display device, overlaying the ideal sweeping trajectory in the user's field of vision in the form of a semi-transparent path, and visualizing the real residual vector as an operation deviation pointing indicator; The haptic feedback unit is used to drive the vibration motor of the handheld device. In response to the user's intention recognition result as input deviation, the vibration intensity of the vibration motor is modulated according to the modulus of the actual residual vector. This embodiment further specifies the method for generating human-computer interaction feedback; the visual enhancement unit renders an operation deviation indicator in the virtual reality headset. This indicator is a dynamic arrow whose starting point is anchored to the virtual nozzle coordinates, and its direction vector... Spatial components of the residual with reality Collinear, length: in, Using the hyperbolic tangent function To prevent visual obstruction, the system overlays the ideal sweeping trajectory as a semi-transparent green pipe, guiding the user to move the fire extinguisher nozzle into the pipe. A haptic feedback unit enhances muscle memory by dynamically modulating the drive voltage of the handheld device's vibration motor based on the weighted modulus of the actual residual. The modulation formula is: in, This indicates taking the smaller of the two values ​​within the parentheses; weighted norm. The calculation formula is: in, , is a weighting coefficient that balances position and posture perception. This coefficient is calibrated through psychophysical experiments so that the posture deviation per unit radian is aligned with the position deviation per unit meter in terms of subjective perceived intensity. The haptic feedback gain coefficient is set to [value]. This value is calculated based on the ratio of the rated drive voltage of the linear vibration motor of the handheld device to the corresponding minimum perceptible deviation modulus, in order to match the physical dimensions of voltage and length. The saturation threshold was set based on ergonomic testing. The corresponding effective correction range for the arm; the further the user deviates from the ideal trajectory, the stronger the vibration output by the system; when they completely overlap, the vibration disappears.

[0030] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A fire extinguishing teaching system based on motion capture, characterized in that, include: The data acquisition module is used to acquire real-time motion capture data streams containing spatial position coordinates, attitude quaternions and acceleration components of the handheld fire extinguisher simulator, as well as status data streams containing the dynamic coordinates of the virtual fire source and fire intensity parameters. The baseline reconstruction module is used to construct an ideal interactive reference space based on the state data stream and calculate the ideal sweeping trajectory covering the root of the virtual fire source as baseline data. The fault simulation module is used to parameterize the preset error mode and inject it into the ideal interactive reference space to generate a labeled theoretical deviation feature library and corresponding multiple sets of parallel simulation trajectories. The difference extraction module is used to calculate the difference between the real-time motion capture data stream and the ideal interactive reference space to generate a real residual vector, and to calculate the difference between each set of parallel simulation trajectories and the ideal interactive reference space to generate a theoretical residual vector set. The topology coupling module is used to map the real residual vector to the feature space of the theoretical residual vector set, calculate the topological similarity between the mapped real residual vector and each vector in the theoretical residual vector set, and select the deviation type corresponding to the highest similarity as the user intent recognition result. The feedback generation module is used to generate feedback instructions based on the user intent recognition results.

2. The fire extinguishing teaching system based on motion capture according to claim 1, characterized in that, The fault simulation module includes: The geometric deviation injection unit is used to simulate the pointing input deviation. It superimposes a preset elevation angle deviation vector or azimuth angle deviation vector on the spatial vector of the ideal sweeping trajectory to generate a parallel simulation trajectory that simulates the aiming offset. The timing distortion unit is used to simulate rhythmic input deviations by compressing or stretching the time axis of the ideal sweeping trajectory to generate a parallel simulation trajectory that simulates abnormal sweeping speed. The noise superposition unit is used to simulate physiological or unstable inputs by superimposing random frequency noise that conforms to muscle fatigue characteristics or specific waveforms that conform to sensor drift characteristics onto the ideal sweeping trajectory to generate a parallel simulation trajectory that simulates hand tremor.

3. The fire extinguishing teaching system based on motion capture according to claim 2, characterized in that, The difference extraction module includes: The synchronization alignment unit is used to perform spatiotemporal alignment of the data before calculating the difference. It uses a dynamic time warping algorithm to map the time sequence of the real-time motion capture data stream onto the time axis of the ideal interactive reference space. The pure feature extraction unit is used to remove reference signal interference. By subtracting the reference data of the ideal sweeping trajectory from the parallel simulation trajectory through vector subtraction, the theoretical residual vector set containing only the decoupled input deviation features is obtained.

4. The fire extinguishing teaching system based on motion capture according to claim 1, characterized in that, The topology coupling module includes: The projection calculation unit is used to perform dimensionality reduction projection of the feature space, projecting the real residual vector and the theoretical residual vector set onto a unified metric space; The similarity decision unit is used to calculate the cosine similarity or Euclidean distance between the projected real residual vector and each of the theoretical residual vectors, and to construct a similarity scoring matrix. The state locking unit is used to lock the dominant input bias pattern according to the similarity score matrix. When the similarity score between the actual residual vector and the theoretical residual vector of a certain type is higher than or equal to the preset confidence threshold, the specific type is determined as the current dominant input bias pattern.

5. The fire extinguishing teaching system based on motion capture according to claim 4, characterized in that, The state locking unit is also configured with noise reduction decision logic: When the similarity score between the actual residual vector and the theoretical residual vector of the simulated aiming offset is higher than or equal to the confidence threshold, and the similarity score between the actual residual vector and the theoretical residual vector of the simulated handshake is lower than or equal to the confidence threshold, the user intent recognition result is determined to be a target pointing error. When the similarity score between the actual residual vector and the theoretical residual vector of the simulated aiming offset is lower than or equal to the confidence threshold, and the similarity score between the actual residual vector and the theoretical residual vector of the simulated handshake is higher than or equal to the confidence threshold, the user intent recognition result is determined to be physiological shaking or insufficient control, and a non-corrective interactive feedback prompting stable operation is generated. When the similarity scores of the actual residual vector, the theoretical residual vector of the simulated aiming offset, and the theoretical residual vector of the simulated handshake are all higher than or equal to the confidence threshold, it is judged as a composite error. Corrective feedback for the target pointing error is output first, and a stable operation prompt is superimposed at the same time. When the similarity scores of the actual residual vector, the theoretical residual vector of the simulated aiming offset, and the theoretical residual vector of the simulated handshake are all lower than or equal to the confidence threshold, it is determined to be an atypical operation or an error mode not defined by the system, and no correction instruction is generated.

6. The fire extinguishing teaching system based on motion capture according to claim 1, characterized in that, The benchmark reconstruction module includes: The physical constraint unit is used to define the motion boundary of virtual object interaction, and sets the activity limit range and maximum movement speed of the virtual representative point according to human kinematic parameters. The optimal path planning unit is used to calculate the sweeping path with the highest interaction efficiency. Based on the dynamic coordinates of the virtual fire source and the current virtual wind direction parameters, the path planning algorithm is used to solve for the ideal sweeping trajectory within the motion boundary that minimizes the time required for the virtual extinguishing agent to cover the root of the fire source.

7. The fire extinguishing teaching system based on motion capture according to claim 1, characterized in that, The feedback generation module includes: The visual enhancement unit is used to overlay and display visual guidance in the virtual reality display device, overlaying the ideal scanning trajectory in the user's field of vision as a semi-transparent path, and visualizing the real residual vector as an operation deviation pointing indicator; A haptic feedback unit is used to drive the vibration motor of the handheld device, and in response to the user intent recognition result as an input deviation, modulates the vibration intensity of the vibration motor according to the modulus of the actual residual vector.

8. The fire extinguishing teaching system based on motion capture according to claim 1, characterized in that, The data acquisition module also includes: The preprocessing unit is used to filter and preprocess the real-time motion capture data stream, using the Kalman filter algorithm to smooth the spatial position coordinates and the attitude quaternions, and to remove outlier data caused by occlusion of the spatial positioning sensor or attitude sensor in the motion capture system.

Images