User operation behavior scoring method and system of simulation training system

Abstract

The invention relates to the technical field of virtual simulation training, solves the technical problems of dynamic coordinate drift and physical-virtual space mapping distortion caused by free rotation of head-mounted equipment, and particularly relates to a user operation behavior scoring method and system of a simulation training system. The method comprises the following steps: establishing an initial mapping relation between a physical coordinate system and a virtual coordinate system on physical equipment; measuring user head motion state data and constructing a pose matrix for describing the head motion; by fusing data of the gyroscope, the accelerometer and the magnetometer, dynamic response and static stability are intelligently balanced, sensor drift is effectively inhibited, and the accuracy of head rotation attitude measurement is ensured; and the acceleration data is converted into millimeter-level displacement by adopting a double integration and motion compensation technology, so that the problem of accumulative errors of traditional inertial navigation is solved, and reliable space translation tracking is realized.

Description

Technical Field

[0001] This invention relates to the field of virtual simulation training technology, and in particular to a user operation behavior scoring method and system for a simulation training system. Background Technology

[0002] As a fundamental industry of the national economy, the mining sector, while providing critical raw materials, also faces severe challenges in safety, efficiency, and talent development. Mine operating environments are typically characterized by high risk, high complexity, variable geological conditions, and the widespread use of heavy equipment. Statistics show that globally, mine accidents remain a major cause of personal injury and property damage. These accidents are often closely related to insufficient operator skills, inadequate emergency response capabilities, or misjudgments of complex working conditions. Furthermore, the gradual adoption of automation and intelligent technologies in the mining sector places higher demands on the technical skills and overall competence of operators.

[0003] Traditional mine training methods, such as classroom lectures, manual reading, apprenticeship, and hands-on practice, have many limitations. While hands-on training is authentic, it is costly and carries extremely high safety risks, especially in the early stages of training, where trainee errors can lead to equipment damage or even personal injury. Furthermore, the complex and ever-changing real-world environment makes it difficult to systematically simulate various abnormal working conditions and emergencies, and trainees lack opportunities to repeatedly practice and handle complex situations in a safe and controlled environment. The apprenticeship system, on the other hand, is easily limited by the mentor's personal experience, time, and energy, and the transfer of knowledge may be inaccurate.

[0004] In recent years, with the rapid development of technologies such as computer graphics, virtual reality, augmented reality, sensor technology, and artificial intelligence, simulation training systems have been increasingly widely used in the mining industry. Simulation training systems provide trainees with a safe, economical, and repeatable training platform by constructing virtual mining operating environments, simulating the operating characteristics of heavy equipment (such as excavators, mining trucks, and tunneling machines), and reproducing various normal and abnormal working conditions. Trainees can practice equipment operation, process simulations, and emergency response in the virtual environment, effectively reducing the risks and costs of on-site training and improving the coverage and efficiency of training.

[0005] However, many current mine simulation training systems still fall short in terms of training effectiveness evaluation. In the field of coal mine safety training, the core challenge of operational behavior scoring technology based on multimodal data stems from the spatiotemporal integration of physical and virtual environments. Particularly in coal mine vertical shaft hoist simulation training systems, when trainees perform critical operations (such as emergency braking), the system needs to simultaneously collect three types of heterogeneous data streams. Specifically, when a trainee completes a closed-loop process within a few seconds of emergency operation—observing the instrument panel, pressing the physical emergency stop button, and operating the virtual brake valve—the physical sensor captures the button press pressure and timing at a high frequency of 1000Hz; the VR system records the spatial trajectory of the controller at a frame rate of 90Hz; and the head-mounted device collects physiological indicators at 120Hz (eye movement) and 1Hz (heart rate). This multi-source synchronous acquisition mechanism introduces a potential for spatiotemporal inaccuracies in subsequent behavior scoring models. Specifically: In mixed reality-based simulation training systems, accurate scoring of user actions relies on precise mapping across spatial coordinate systems. Current technical architectures commonly suffer from the following spatial dimension deficiencies: geometric mapping discrepancies between the physical device coordinate system and the virtual environment coordinate system (e.g., the discrepancy between the Cartesian coordinate system and the model coordinate system), and non-rigid coordinate drift caused by the dynamic displacement of the head-mounted device, resulting in additional defects in real-world operations. When a user's head rotates freely, the optical capture coordinate system of the tracking device dynamically shifts from the global reference coordinate system. This shift is not a static error, but a "drift" process that changes continuously with the movement. During training, when trainees perform positioning operations (such as looking at the dashboard or aligning with a virtual brake valve), the accuracy of the initial calibration rapidly diminishes due to minute head movements. The system incorrectly attributes the user's effective visual focus or limb pose as an invalid area or deviation from the target, and this drift is gradual and irreversible. As a result, the operational accuracy scoring model exhibits a systematic bias in the spatial dimension—originally high-precision, standard operations are repeatedly misjudged as positional deviations (such as false "operational errors"), thereby reducing the confidence of the entire evaluation system. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a user operation behavior scoring method and system for a simulation training system. It solves the technical problems of dynamic coordinate drift and physical-virtual space mapping distortion caused by the free rotation of head-mounted devices, achieves millimeter-level spatial consistency maintenance of operation posture, and completely eliminates systematic misjudgments in training scoring.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a user operation behavior scoring method for a simulation training system, the method comprising the following steps: Establish the initial mapping relationship between the physical coordinate system and the virtual coordinate system on the physical device; Monitor user head movement data and construct a pose matrix describing head movement; A feedforward mapping matrix is ​​generated based on the pose matrix and the initial spatial mapping relationship to compensate for coordinate system drift caused by head movement in real time. Residual mapping errors are detected by reference anchor points in a virtual coordinate system, spatial error vectors are calculated, and feedforward mapping matrix is ​​dynamically optimized to generate the final mapping matrix. The physical operation coordinates are converted into virtual space coordinates using the final mapping matrix, and the user's operation behavior is comprehensively scored by combining the credibility weight.

[0008] Furthermore, establishing the initial mapping relationship between the physical coordinate system and the virtual coordinate system on the physical device includes: Physical markers are deployed on the surface of physical devices to form a set of physical markers, and a set of virtual anchors corresponding to the physical markers is created in the virtual environment. Calculate the affine transformation matrix that minimizes the error between the physical markers and the virtual anchors, i.e.: In the formula, is the optimal coordinate transformation matrix from the physical coordinate system to the virtual coordinate system; M is the affine transformation matrix to be solved; V is the point cloud matrix of the virtual anchor points; P is the point cloud matrix of the physical marker points; F is the identifier of the Frobenius norm, defined as the square root of the sum of the squares of all elements of the matrix; Next, calculate the residual vector used to compensate for assembly errors, i.e.: In the formula, The residual vector; Based on the optimal coordinate transformation matrix and residual vector, an initial mapping relationship between the physical and virtual coordinate systems is established to reduce the mapping error to the hardware limit, namely: In the formula, Set the coordinates in the virtual coordinate system; The coordinates are in the physical coordinate system; T is the transpose operator. Verify the global accuracy of the initial mapping relationship using uncalibrated points.

[0009] Furthermore, the verification of the global accuracy of the initial spatial mapping relationship through uncalibrated points includes: Select a set of test points on the physical device that were not calibrated, substitute their coordinates into the optimal coordinate transformation matrix, calculate the virtual projection position, and compare the deviation between the actual virtual coordinates and the projected coordinates, i.e.: In the formula, This represents the spatial mapping deviation value for the i-th uncalibrated point; Let be the true theoretical coordinates of the i-th uncalibrated point in the virtual coordinate system; The projected coordinates of the i-th uncalibrated point are calculated using the mapping relationship; The spatial mapping deviation value of the uncalibrated points is compared with the system tolerance threshold. When the spatial mapping deviation value is greater than the system tolerance threshold, a residual compensation field is constructed to eliminate nonlinear distortion, i.e.: In the formula, This is the residual compensation vector at the point to be compensated; The physical coordinates of the point to be compensated; Let i be the coordinates of the i-th physical marker point; The weight coefficient for the i-th physical marker point; Gaussian kernel function; Furthermore, the topology of the device skeleton used for deformation propagation analysis is extracted, and the local deformation of the physical device is dynamically mapped to the precise compensation displacement of the target node in the virtual space, i.e.: In the formula, Let be the deformation compensation vector of the j-th key structural point in virtual space; Let j be the set of neighbor nodes of the j-th key structural point; Let be the deformation transfer coefficient from the i-th critical structural point to the j-th critical structural point; Let be the measured physical deformation direction of the i-th neighbor node; Generate a deformation Jacobian correction matrix for dynamically adjusting coordinate transformation, convert the deformation compensation vector into matrix operation form, and output the adjusted coordinate transformation matrix; and ensure that the global error probability converges within the system tolerance threshold by randomly sampling verification points.

[0010] Furthermore, the monitoring of user head movement state data and the construction of a pose matrix describing head movement include: The state data includes the velocity angle vector, acceleration vector, and magnetic field strength vector; A reference quaternion is generated based on the acceleration vector and the magnetic field strength vector; Calculate attitude quaternions based on velocity angle vectors; The weighted fusion of reference quaternions and attitude quaternions is used to output a fused attitude quaternion for the final head attitude in spatial localization. The true translational acceleration used to transform local coordinate system acceleration to physical coordinate system acceleration is calculated by fusing attitude quaternions and acceleration vectors. In the formula, This is the original acceleration vector in the physical coordinate system; To be the inverse quaternion of the fusion attitude quaternion; It is the acceleration vector; The true motion acceleration vector, which eliminates the interference of gravity on the Z-axis acceleration, is calculated using the original acceleration vector and the unit gravity vector. In the formula, This represents the actual acceleration vector. It is the unit vector of gravity; The real motion acceleration vector is fused with the historical real motion acceleration vector using a time-varying weighted method, that is: In the formula, This is the filtered acceleration output vector; This represents the actual acceleration vector at the current moment. This is the filtered acceleration output vector from the previous moment; For adaptive Kalman gain; The displacement vector is calculated by double integration of the filtered acceleration data, i.e.: ; In the formula, in the formula, It is the displacement vector; A homogeneous transformation matrix is ​​constructed by combining the displacement vector and the attitude matrix, and a corrected transformation matrix that is synchronized with the actual physical motion is output by linear interpolation. Select reference marker points to perform bidirectional coordinate transformation to verify the transformation fidelity of the correction transformation matrix. If the correction transformation matrix does not meet the transformation fidelity, correct the calculated displacement vector. The pose matrix is ​​constructed based on the attitude matrix and the corrected optimal displacement vector.

[0011] Furthermore, the weighted fusion of the reference quaternion and the attitude quaternion is performed as follows: The projection of the gravity direction from the physical coordinate system onto the local coordinate system is calculated using attitude quaternions, i.e.: In the formula, This is the theoretical direction vector of gravity. This is the standard gravity vector in the physical coordinate system; For attitude quaternions; It is the inverse quaternion of the attitude quaternion; The drift error of the quantized gyroscope attitude in the gravity direction, reflecting the attitude reliability, is calculated using attitude quaternions, the theoretical gravity direction vector, and the gravity unit vector. In the formula, This is the error vector for the direction of gravity. This is the gradient function of the gravity error; The magnitude of the gravity direction error vector, a preset upper limit for static weights, and a dynamic sensitivity coefficient are used to calculate an adaptive balance between static drift suppression and dynamic response preservation, i.e.: In the formula, For fusion weighting coefficients; This represents the upper limit of static weights. This refers to the dynamic sensitivity coefficient. Let be the magnitude of the gravity direction error vector; The formula for calculating the fused attitude quaternion is: In the formula, To fuse pose quaternions; For reference quaternions; It is a quaternion of attitude.

[0012] Furthermore, the step of selecting reference marker points to perform bidirectional coordinate transformation to verify the transformation fidelity of the correction transformation matrix includes: A bidirectional transformation is performed on the reference marker point. The forward transformation maps the physical coordinates of the reference marker point to the local coordinate system of the current head, and the inverse transformation remaps the result back to the physical coordinate system. The bidirectional coordinate mapping residual is then calculated. In the formula, Use as a reference marker point; This is a forward transformation operation; This is the inverse transformation operation; For coordinate bidirectional mapping residuals; The coordinate bidirectional mapping residual is compared with a preset residual threshold. If the coordinate bidirectional mapping residual is greater than the residual threshold, the displacement vector is corrected during calculation. In the formula, This is the corrected optimal displacement vector; Let represent the three-dimensional displacement vector to be solved, where Represents the set of real numbers in three-dimensional space; IMU confidence weights; The displacement vector calculated by the IMU is: The displacement vector for optical positioning is: In the formula, Let be the three-dimensional coordinates of the i-th marker point in the local coordinate system; Indicates the actual observed value; For projection functions; Let be the homogeneous transformation matrix containing the three-dimensional displacement vector to be solved and a fixed attitude matrix; where This is the attitude matrix.

[0013] Furthermore, the method for calculating the spatial error vector and dynamically optimizing the feedforward mapping matrix is ​​as follows: Define the theoretical gaze area of ​​a fixed reference object; The user gaze vector output by the eye-tracking device is projected onto the virtual space through a feedforward mapping matrix. The user's initial gaze coordinates in the virtual space are obtained, and transient noise is filtered to output the effective gaze coordinates after clear denoising of biomechanical features. Calculate the projection deviation between the effective gaze point coordinates after denoising and the center coordinates of the reference anchor point, i.e.: In the formula, This is the spatial error vector; The center coordinates of the reference anchor point; These are the effective gaze point coordinates after denoising. The feedforward mapping matrix is ​​updated by moving the spatial error vector along the direction of error reduction, i.e.: In the formula, This is the optimized feedback mapping matrix; For matrix gradient operators; For adaptive step size.

[0014] Furthermore, the method for generating the final mapping matrix is ​​as follows: The dynamic weight coefficients of the feedback mapping matrix and feedforward mapping matrix after dynamic fusion optimization are calculated using the spatial error vector, i.e.: In the formula, These are dynamic weighting coefficients; The characteristic scale constant; The formula for calculating the final mapping matrix is: In the formula, This is the final mapping matrix; This is the feedforward mapping matrix.

[0015] Furthermore, the comprehensive scoring of user behavior based on credibility weights includes: The final mapping matrix is ​​used to perform an instantaneous matrix transformation calculation on the physical markers in the physical marker set, that is: In the formula, Let i be the coordinates of the i-th physical marker point; The corrected virtual coordinates of the i-th physical marker point in the virtual coordinate system; The dynamic spatial residual used to quantify spatial mapping errors is calculated by correcting the virtual coordinates and the original measurements from the VR system. For dynamic space residuals; These are the original measurement values ​​from the VR system. Based on the dynamic spatial residual and a preset error tolerance threshold, physical spatial distortion is converted into scoring confidence, i.e.: In the formula, As a credibility weight; This is the preset error tolerance threshold; d is the preset error failure threshold; d is the attenuation slope coefficient. By adding a credibility weight to the existing scoring model, a comprehensive quantitative evaluation result of user behavior is output, namely: In the formula, Score the final behavior; The original spatial score generated based on spatial coordinate data; Logical scoring.

[0016] A user behavior scoring system for a simulation training system includes: The initial spatial mapping module is used to establish the coordinate system alignment between physical devices and the virtual environment; The head motion tracking module is used to capture head movement in real time and generate pose data; The dynamic drift compensation module is used to eliminate coordinate drift caused by head movement; Spatial error optimization module, used to correct mapping distortion in real time; The behavior scoring module is used to comprehensively evaluate user actions.

[0017] By employing the above technical solution, the present invention provides a user operation behavior scoring method and system for a simulation training system, which has at least the following beneficial effects: 1. This invention integrates data from gyroscopes, accelerometers, and magnetometers to intelligently balance dynamic response and static stability, effectively suppressing sensor drift and ensuring the accuracy of head rotation posture measurement.

[0018] 2. This invention employs dual integration and motion compensation technology to convert acceleration data into millimeter-level displacement, solving the problem of accumulated error in traditional inertial navigation and achieving reliable spatial translation tracking.

[0019] 3. This invention establishes a closed-loop verification mechanism, which dynamically corrects mapping errors through bidirectional coordinate transformation of reference marker points, maintaining high-precision alignment between virtual space and physical space.

[0020] 4. This invention utilizes a predictive compensation algorithm to eliminate the effects of system latency, ensuring that coordinate transformation is synchronized with the user's actual head movement at the millisecond level, thus avoiding motion lag in virtual reality interaction.

[0021] 5. This invention combines the high-frequency motion trend of the IMU with the absolute reference of optical positioning, and maintains a stable spatial mapping relationship even when the sensor fails by intelligently weighting and optimizing the displacement vector.

[0022] 6. From static fine-tuning operations to rapid head movements, the system can automatically adjust filtering parameters and fusion weights to adapt to the accuracy requirements of different usage scenarios. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the scoring method in the implementation of the present invention. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding and implementation of how the present application uses technical means to solve technical problems and achieve technical effects.

[0025] This embodiment proposes a user operation behavior scoring method for a simulation training system. By correcting coordinate drift caused by head movements in real time, it achieves high-precision spatial positioning and comprehensive scoring of user operation behavior in simulation training. Figure 1 As shown, the method includes the following steps: Infrared laser reflective markers are evenly distributed on the surface of key operating components (such as buttons and joysticks) of physical devices. These physical markers form a physical reference network, generating a set of physical markers. Its three-dimensional coordinates By pre-measuring and storing data using a high-precision laser rangefinder, physical spatial anchor points that are unaffected by environmental interference are formed, thereby establishing an absolute position reference system in physical space, i.e., a physical coordinate system. Here, n represents the number of physical marker points.

[0026] Create virtual anchor points that are isomorphic to physical marker points in the virtual environment and merge them into a set of virtual anchor points. Its position coordinates Generated strictly according to the physical device model; where m is the number of virtual anchor points, and m=n, indicating that each physical marker point There must be one and only one corresponding virtual anchor point. And a topological constraint algorithm is used to ensure that the relative distance between virtual anchor points and the distance between physical marker points satisfy the following: In the formula, i and j are index numbers; Let be the Euclidean distance between the i-th physical marker and the j-th physical marker; Let be the Euclidean distance between the i-th virtual anchor point and the j-th virtual anchor point.

[0027] By constructing a virtual coordinate system that completely corresponds to the physical marker points, the initial deformation error caused by model rendering can be eliminated, achieving geometric consistency between physical and virtual spaces.

[0028] The affine transformation matrix that minimizes the error between the physical and virtual coordinate systems is obtained by using the singular value decomposition algorithm, i.e.: In the formula, The optimal coordinate transformation matrix from the physical coordinate system to the virtual coordinate system describes the spatial transformation relationship between the physical and virtual coordinate systems. It is used to realize rigid transformations such as rotation and scaling, and to accurately map the physical operation pose to the virtual environment. M is the affine transformation matrix to be solved (3×3); V is the point cloud matrix of the virtual anchor points (m×3), which stores the theoretical coordinates of m virtual anchor points; P is the point cloud matrix of the physical marker points (n×3), which stores the measured coordinates of n physical marker points. F is the square of the matrix norm. It quantifies the overall mapping error intensity between the point cloud matrix of physical markers and the point cloud matrix of virtual anchors after affine transformation by calculating the sum of the squares of all elements of the matrix. Its minimization process ensures that the coordinate system transformation achieves optimal rigid alignment in the entire space. F is the identifier of the Frobenius norm, which is defined as the square root of the sum of the squares of all elements of the matrix.

[0029] Simultaneously calculate the residual vector used to compensate for assembly errors, i.e.: In the formula, The residual vector is essentially the position deviation matrix after mapping from physical space to virtual space. The optimal coordinate transformation matrix can only eliminate rigid transformation errors (rotation / scaling), while the residual vector can capture all errors that rigid transformations cannot describe. It quantifies the actual position deviation between the point cloud matrix of physical markers and the point cloud matrix of virtual anchor points after mapping by the optimal coordinate transformation matrix, and stores the 3D coordinate corrections for all physical markers. ;in, This refers to the horizontal compensation amount; This is the vertical height compensation amount; This is the amount of compensation for depth distance.

[0030] Based on the optimal coordinate transformation matrix and residual vector, an initial mapping relationship between the physical and virtual coordinate systems is established to reduce the mapping error to the hardware limit, namely: In the formula, The coordinates (column vector) in the virtual coordinate system represent the final positions mapped to the virtual scene. The coordinates (column vector) in the physical coordinate system reflect the actual spatial location of the operation point; T is the transpose operator, used to convert a row vector into a column vector.

[0031] Then through uncalibrated points Verify the global accuracy of the mapping relationship: Select a set of test points on the physical device that were not calibrated. Substitute its coordinates into the optimal coordinate transformation matrix and calculate the virtual projection position. By comparing the deviation between the actual virtual coordinates and the projected coordinates, i.e.: In the formula, This represents the spatial mapping deviation value for the i-th uncalibrated point; Let be the true theoretical coordinates of the i-th uncalibrated point in the virtual coordinate system; The projected coordinates of the i-th uncalibrated point are calculated using the mapping relationship.

[0032] Compare the spatial mapping deviation value of the uncalibrated points with the system tolerance threshold. If the following conditions are met: In the formula, The maximum spatial mapping deviation value among all uncalibrated points; The system tolerance threshold is determined based on fitting experimental data.

[0033] This indicates that the spatial mapping system has passed robustness verification, and the mapping error is less than the system tolerance threshold in the entire operating space (including edge regions); the residual compensation model can be effectively generalized to non-standard fixed points, and thus subsequent stage operations can be performed to ensure the credibility of the data.

[0034] If these conditions are not met, it indicates that the initial spatial calibration failed, and the system exhibits uncompensated nonlinear distortions. These distortions could be caused by excessive physical equipment assembly tolerances, virtual model rendering deformation, micro-deformation of metal components due to changes in environmental temperature and humidity, or gradual displacement due to stress release in the mechanical structure. Therefore, it is necessary to transform the failed rigid calibration system into a dynamic mapping system that can adapt to environmental changes, completely eliminating virtual-physical spatial misalignment caused by structural distortions and time-varying interference. Specifically, the following methods can be used: Calculate the residual vector of all physical markers in the physical marker set, and construct a radial basis function interpolation field (i.e., residual compensation field) that generalizes the discrete calibration error to a global continuous vector field, namely: In the formula, The residual compensation vector at the point to be compensated represents the positional error that needs to be compensated at that point. The physical coordinates of the point to be compensated; Let i be the coordinates of the i-th physical marker point; The weight coefficient for the i-th physical marker point is used to control the influence of this reference point on the entire domain, and is obtained by solving the least squares method. It is a radial basis function, i.e., a Gaussian kernel function.

[0035] Extract the skeleton topology of physical devices Where H represents key structural points (such as button hinges and panel hinges); E represents rigid body connections. By learning the deformation propagation law through a graph neural network, the local deformation of the physical device is dynamically mapped to the precise compensation displacement of target nodes in virtual space, i.e.: In the formula, Let be the deformation compensation vector of the j-th key structural point in the virtual space, representing the amount of displacement that needs to be compensated in the virtual space due to physical deformation; This is the set of neighboring nodes of the j-th critical structural point, used to define the deformation propagation range (e.g., components directly bolted to the j-th critical structural point). The deformation transfer coefficient from the i-th critical structural point to the j-th critical structural point is determined by the material properties of the equipment. Let be the measured physical deformation vector of the i-th neighbor node.

[0036] The physical coordinates of the points to be compensated are subjected to forward-inverse joint compensation to dynamically eliminate nonlinear distortion. The forward compensation is as follows: In the formula, For the virtual space coordinates corrected with compensation points, the physical device coordinates are converted into high-precision virtual positions in real time through algorithm compensation, eliminating the static error of the initial mapping and ensuring high-precision alignment between the physical and virtual spaces; This is the adjusted coordinate transformation matrix; This is a matrix combination operator used to convert deformation compensation quantities into matrix operation forms; This is the deformation Jacobian correction matrix, used for dynamic adaptation: converting the deformation compensation vector into a linear correction of the coordinate transformation.

[0037] B verification points are randomly sampled in the physical coordinate system. Probabilistic convergence is used to determine the accuracy across the entire space, ensuring that the global error is consistently below a threshold. If the threshold is not met, automatic closed-loop iterative optimization is performed. If the conditions are not met, the residual compensation field parameters are automatically adjusted and returned, forming a closed-loop optimization.

[0038] Three sets of 9-axis inertial measurement units (IMUs) are deployed on the rigid structure of the head-mounted device (top of the helmet / left and right forehead beams) to collect angular velocity, linear acceleration, and magnetic field data of head movements from all directions; each IMU outputs a raw data stream, including: In the formula, The velocity angular vectors represent the angular velocity components of the head's rotation around the three axes of the local coordinate system (with the physical center of the head-mounted device as the origin). These components are directly acquired by the gyroscopes in the IMU, reflecting the instantaneous rotational state. This represents the pitch angular velocity around the horizontal axis of the head-mounted equipment (the direction of the line connecting the left and right ears); This represents the yaw rate about the longitudinal axis (head-chin direction); It represents the rolling angular velocity about the vertical axis (from the tip of the nose to the back of the head); This is the acceleration vector, reflecting the linear acceleration components along the three axes in the local coordinate system, obtained through accelerometer measurements; where... This is the acceleration along the horizontal axis (left-right direction); This is the acceleration along the vertical axis (up and down direction); This is the acceleration along the vertical axis (front-to-back direction); Let be the magnetic field strength vector, representing the three-axis components of the geomagnetic field strength in the local coordinate system, obtained through magnetometer measurements; where The horizontal axis magnetic field component; The vertical axis magnetic field component; This represents the vertical axis magnetic field component. The underlying data capture layer for head motion is formed through the raw data stream, providing the hardware foundation for drift calculations.

[0039] By normalizing the acceleration vector, a gravity unit vector is obtained to establish the vertical reference axis; then, by normalizing the magnetic field strength vector, a magnetic north unit vector is obtained to establish the horizontal reference. Based on the cross product of the gravity unit vector and the magnetic north unit vector, the eastward vector used to establish the east axis of the right-hand coordinate system is calculated, i.e.: In the formula, It is an eastward vector; It is the unit vector of gravity; This is the magnetic north unit vector.

[0040] Then, based on the cross product direction of the unit gravity vector and the eastward vector, the northward vector used to correct the magnetic north vector deviation is calculated, that is: In the formula, This is the northward vector.

[0041] The eastward vector, northward vector, and gravity unit vector are combined into an orthogonal basis matrix, i.e., a rotation matrix, for constructing the mathematical representation of spatial transformations. This describes the rotation transformation from the physical coordinate system to the local coordinate system, providing an orthonormal basis for quaternion transformations. It converts the rotation matrix into a quaternion form that can directly participate in fusion calculations, i.e.: In the formula, This is a reference quaternion generated by fusing accelerometer and magnetometer data, used to provide a reference for drift correction of gyroscope integration; This is a matrix-to-quaternion function.

[0042] The velocity angle vector is normalized to extract the original angular velocity vector at the current moment, representing the pure direction information of the rotational motion. The instantaneous change in the head rotation attitude is then calculated in real time using this original velocity angle vector. In the formula, The attitude quaternion after integration at the current time step; The attitude quaternion of the previous time step describes the rotation state of the local coordinate system relative to the physical coordinate system at time t-1; t is the time scale. This is a quaternion multiplication operator, representing the superposition operation of rotated states; The incremental rotation quaternion describes the instantaneous rotation angle within the sampling time interval; where, This is the original angular velocity vector; Let be the angular displacement, and let be the integral of the velocity angle over time, i.e.: In the formula, This represents the sampling time interval.

[0043] The projection of the gravity direction from the physical coordinate system onto the local coordinate system is calculated using attitude quaternions, i.e.: In the formula, This is the theoretical direction vector of gravity. This is the standard gravity vector in the physical coordinate system; For attitude quaternions; is the inverse quaternion of the attitude quaternion, representing the inverse transformation of the rotation operation.

[0044] The drift error of the quantized gyroscope attitude in the direction of gravity is calculated using attitude quaternions, the theoretical gravity direction vector, and the gravity unit vector, reflecting the attitude reliability. In the formula, This is the error vector for the direction of gravity. The gradient function of gravity error is used to quantize the sensitivity of the attitude quaternion to gravity direction error.

[0045] The magnitude of the gravity direction error vector, a preset upper limit for static weights, and a dynamic sensitivity coefficient are used to calculate an adaptive balance between static drift suppression and dynamic response preservation, i.e.: In the formula, The fusion weighting coefficients reflect the weight of the accelerometer data in the fusion process; The static weight upper limit represents the baseline fusion weight when the head-mounted device is stationary. It is a constant in the interval (0, 1) and is calibrated through experimental data. The dynamic sensitivity coefficient is a constant greater than 0, used to control the switching speed from static to dynamic, and is determined based on fitting experimental data. The magnitude of the gravity direction error vector quantifies the gyroscope attitude prediction error.

[0046] By fusing the weighted attitude quaternion with weighted coefficients and the reference quaternion, a dynamically weighted fused attitude quaternion is output, which represents the low-frequency stability of the accelerometer and the high-frequency response of the gyroscope. In the formula, To fuse posture quaternions, namely interference-resistant head rotation quaternions, to accurately describe changes in head orientation.

[0047] The true translational acceleration used to transform local coordinate system acceleration to physical coordinate system acceleration is calculated by fusing attitude quaternions and acceleration vectors. In the formula, This is the original acceleration vector in the physical coordinate system. To eliminate the centrifugal acceleration caused by head rotation, the pure translation component is retained. The inverse quaternion of the fused attitude quaternion represents the reverse rotation relation.

[0048] The true motion acceleration vector, which eliminates the interference of gravity on the Z-axis acceleration, is calculated using the original acceleration vector and the unit gravity vector. In the formula, This is the true acceleration vector, eliminating the interference of gravity on acceleration measurement, thus separating the true acceleration generated purely by head translation.

[0049] An adaptive Kalman filter is used to dynamically calculate and perform time-varying weighted fusion of the real motion acceleration vector and the historical real motion acceleration vector, i.e.: In the formula, The filtered acceleration output vector reflects the effective motion acceleration after noise suppression; This represents the actual acceleration vector at the current moment. The filtered acceleration output vector from the previous moment is used to provide a reference for motion continuity. The adaptive Kalman gain, used to balance the confidence levels of current measurements and historical data, is dynamically adjusted based on the gyroscope / accelerometer noise covariance to suppress high-frequency vibration noise. The calculation formula is as follows: In the formula, The accelerometer noise variance is obtained by calculating the triaxial acceleration variance within a sliding time window (e.g., 50ms). The gyroscope noise variance is obtained by calculating the variance of the three-axis angular velocity within the same time window.

[0050] The filtered acceleration output vector is subjected to a double integration operation. The first integration yields the instantaneous velocity loss, and the second integration yields the three-dimensional translational displacement of the head in the physical coordinate system, which is used to quantize the displacement. In the formula, Let be the displacement vector, representing the displacement from the initial time. The cumulative head displacement up to the current time t; ,in , , These correspond to pure translational displacements along the X-axis (left / right), Y-axis (up / down), and Z-axis (back / forward) in the physical coordinate system, respectively. By converting the filtered acceleration output vector into displacement data with millimeter-level precision, the coordinate drift problem in traditional methods is solved.

[0051] The fused attitude quaternions are converted into attitude matrices that can be directly computed by the space mapping engine, eliminating the singularity problem during the transformation from fused attitude quaternions to Euler angles and maintaining the numerical stability of the rotation description. The fused attitude matrix and displacement vectors generate a homogeneous transformation matrix, i.e.: In the formula, It is a homogeneous transformation matrix. represents a matrix element in the attitude matrix.

[0052] Collect homogeneous transformation matrices aligned with consecutive timestamps and construct them into a sequence of transformation matrices, i.e.: in, Let be the homogeneous transformation matrix of the previous sampling time. This is the homogeneous transformation matrix predicted for future times.

[0053] By applying linear interpolation, the transformation matrix at the current moment is weighted and fused with the predicted matrix at future moments to dynamically generate a corrected transformation matrix synchronized with the actual physical motion, i.e.: In the formula, To correct the transformation matrix; These are the adjustment coefficients used to control the intensity of the prediction compensation. Predictive compensation ensures millisecond-level synchronization between the homogeneous transformation matrix and the user's actual head movement, thereby avoiding misjudgments of coordinate drift caused by system latency.

[0054] Select any physical marker point as a reference marker point in the physical coordinate system. This reference marker point is a high-precision spatial reference point with an invariant physical location and known coordinates. Perform a bidirectional transformation on the reference marker point: first, map the physical coordinates of the reference marker point to the local coordinate system of the current head through a forward transformation; then, remap the result back to the physical coordinate system through a reverse transformation. Calculate the bidirectional coordinate mapping residual, i.e.: In the formula, Use as a reference marker point; For a positive transformation operation, the reference marker point is mapped to a new coordinate position in the head-mounted device's current local coordinate system by correcting the transformation matrix; This is a mathematical operation that reverses the transformation, transforming the coordinates of a point in the local coordinate system back to the physical coordinate system. The coordinate bidirectional mapping residual reflects the final error of the coordinate system bidirectional transformation and is a core indicator for quantifying the transformation fidelity of the correction transformation matrix.

[0055] The coordinate bidirectional mapping residual is compared with a preset residual threshold. If the coordinate bidirectional mapping residual is less than the residual threshold, it indicates that the current modified transformation matrix satisfies the mathematical closure property, and the error of the current displacement vector is within the residual threshold. Therefore, no correction is needed to ensure the reliability of the spatial mapping. If the coordinate bidirectional mapping residual is greater than the residual threshold, it indicates that the displacement vector of the current modified transformation matrix has out-of-range distortion and cannot satisfy the mathematical closure law of rigid transformation. The spatial mapping relationship must be corrected immediately, and the displacement vector must be recalculated. In the formula, The corrected optimal displacement vector eliminates drift caused by single sensor failure and ensures the reliability of virtual space mapping. Let represent the three-dimensional displacement vector to be solved, where Represents the set of real numbers in three-dimensional space; IMU confidence weights; The displacement vector calculated by the IMU is used to provide high-frequency displacement trends; that is: The displacement vector for optical positioning is used to provide an absolute spatial reference, i.e.: In the formula, Let be the three-dimensional coordinates of the i-th marker point in the local coordinate system; This represents the actual observed value, reflecting the true 2D pixel coordinates of the i-th marker point in the local coordinate system within the camera, and is directly output by the optical tracking system. This is a projection function used to map physical coordinates onto the 2D imaging plane of the camera, simulating the actual optical imaging process (including lens distortion model). Let be the homogeneous transformation matrix containing the three-dimensional displacement vector to be solved and a fixed attitude matrix; where This is the attitude matrix.

[0056] The modified transformation matrix is ​​corrected by the modified optimal displacement vector to obtain the updated transformation matrix. The coordinate bidirectional mapping residual is then recalculated using the updated transformation matrix to ensure that the coordinate bidirectional mapping residual is less than the residual threshold.

[0057] A pose matrix describing the rigid body changes of head motion is constructed based on the attitude matrix and the corrected optimal displacement vector, i.e.: In the formula, This is the pose matrix for head movement, used to map points in the local coordinate system to the physical coordinate system, and outputs the coordinate system offset caused by head movement in real time.

[0058] Dynamic mapping synthesis is performed on the pose matrix and the optimal coordinate transformation matrix to generate a feedforward mapping matrix used to compensate for coordinate system drift caused by user head movements in real time, i.e.: In the formula, It is a feedforward mapping matrix used to offset the coordinate system drift caused by the user's head movement in real time, so that the virtual operation interface remains visually unchanged. It is the inverse of the pose matrix, used to mathematically offset the coordinate shift caused by head movement.

[0059] Using pre-set fixed reference objects in the virtual environment (such as static high-precision models of dashboards and valve operating points) as reference anchor points, residual mapping errors caused by non-rigid deformation are detected by comparing the user's actual gaze position with the theoretical projection position of the anchor point. Specifically: Define the theoretical gaze area of ​​a fixed reference object, namely: In the formula, The tolerance domain, which serves as the reference anchor point, is the spherical detection region constructed in virtual space. Centered on A closed sphere with radius : the core spatial criterion used to determine whether the user's gaze point is within the effective detection range; where The center coordinates of the reference anchor point, The allowable gaze tolerance radius.

[0060] The user's gaze vector, output by the eye-tracking device, is projected onto the virtual space using a feedforward mapping matrix to generate a precise position representing the user's current gaze in the virtual environment. In the formula, The initial gaze coordinates of the user in the virtual space; This is the user gaze vector.

[0061] Then, transient noise is filtered out from the initial gaze point coordinates using a sliding window mean filter, outputting a stable gaze signal with clear biomechanical characteristics, i.e.: In the formula, represents the effective gaze point coordinates after denoising; N is the number of effective samples within the sliding window, N=k+1; is the initial gaze point coordinate of the i-th frame; k is the time backtracking depth, used to control the time span parameter of the sliding window.

[0062] Calculate the projection deviation between the effective gaze point coordinates after denoising and the center coordinates of the reference anchor point, i.e.: In the formula, The spatial error vector contains the direction (vector) and magnitude (scalar) of the positional deviation, and is used to quantify the residual spatial distortion after feedforward compensation.

[0063] Furthermore, it extracts the angular and distance errors from the spatial error vector to quantify the difference between the user's line of sight and the standard operating viewpoint, ensuring that trainees observe key equipment in a standardized posture (e.g., vertically facing the instrument panel), thus eliminating misreading and misjudgment caused by tilted viewing angles. In the formula, This is the directional deviation angle, used to identify non-standard viewpoints; This is the distance error, used to detect visual focus shift; It is the inverse cosine function; is the unit normal vector of the reference surface.

[0064] Simultaneously, it checks whether the coordinates of the effective gaze point fall within the tolerance range of the reference anchor point, i.e.: If the output is not set to the spatial error vector, then the output is frozen.

[0065] By updating the feedforward mapping matrix along the direction of error reduction using the spatial error vector, the spatial alignment error is converged to a threshold imperceptible to the human eye. In the formula, The optimized feedback mapping matrix, containing updated rotation / translation parameters, can more accurately map physical space points to the virtual environment; It is a matrix gradient operator used to accurately quantify the influence of each parameter (rotation / translation component) of the feedforward mapping matrix on spatial error, providing a mathematical basis for coordinate mapping correction; This is an adaptive step size, used to dynamically adjust and optimize the stride length, i.e.: In the formula, The confidence coefficient of the sensor is obtained by dynamically calculating the signal-to-noise ratio and calibration residuals of IMU and eye-tracking device data in real time.

[0066] The dynamic weight coefficients of the feedback mapping matrix and feedforward mapping matrix after dynamic fusion optimization are calculated using the spatial error vector, i.e.: In the formula, These are dynamic weighting coefficients used to control the fusion ratio of the feedforward mapping matrix and the feedback mapping matrix; The characteristic scale constant is a preset constant derived from the system's physical constraints and human-computer interaction characteristics.

[0067] The optimized feedback mapping matrix and feedforward mapping matrix are weighted and fused using dynamic weighting coefficients to output the final mapping matrix used for real-time conversion of physical space coordinates to virtual space coordinates, i.e.: In the formula, This is the final mapping matrix, with drift-resistant spatial mapping parameters.

[0068] The final mapping matrix is ​​used to perform instantaneous matrix transformation calculations on the physical markers in the physical marker set, outputting high-fidelity virtual coordinates, i.e.: In the formula, Let i be the coordinates of the i-th physical marker point; Let be the corrected virtual coordinates of the i-th physical marker point in the virtual coordinate system, representing the precise projection position of the physical operation point in the virtual environment.

[0069] The dynamic spatial residual used to quantify spatial mapping errors is calculated by correcting the virtual coordinates and the original measurements from the VR system. The dynamic spatial residual, i.e., the scalar value of the spatial mapping error at the current moment, quantifies the degree of deviation between the corrected position and the actual measured position in the virtual coordinate system; These are the original measurement values ​​for the VR system.

[0070] Based on dynamic spatial residuals and preset error tolerance thresholds, physical spatial distortion is transformed into scoring confidence, generating continuous spatial credibility and quantifying the reliability of the current coordinates. In the formula, This is the credibility weight, which ranges from [0, 1]. The larger the value, the higher the credibility. This is the preset error tolerance threshold, i.e., the accuracy safety boundary of spatial mapping; is the preset error failure threshold, representing the critical point at which the spatial mapping completely fails; d is the decay slope coefficient, used to control the rate of decrease of the confidence weight in the transition region.

[0071] By adding a credibility weight to the existing scoring model, a comprehensive quantitative evaluation result of user behavior is output, namely: In the formula, The final behavior score is a robust behavior assessment decision. The raw spatial score is generated based on spatial coordinate data and includes dimensions such as operational accuracy (e.g., the distance of the handle position from the target area), visual focus accuracy (e.g., the overlap rate between the gaze point and the key instrument) and pose regularity (e.g., body posture angle error). The logical scoring includes dimensions such as the completeness of the operation sequence (e.g., the correctness of the order of "emergency stop button → brake valve"), the reasonableness of the response time (e.g., whether the delay from alarm to operation is within the threshold), the consistency of equipment status (e.g., whether the change in virtual pressure gauge value after operation is as expected) and physiological indicators (e.g., whether the change in heart rate matches the intensity of emergency operation).

[0072] This embodiment also proposes a user operation behavior scoring system for a simulation training system. The system includes an initial space mapping module, a head motion tracking module, a dynamic drift compensation module, a spatial error optimization module, and a behavior scoring module.

[0073] The initial space mapping module is used to establish the coordinate system alignment between physical devices and the virtual environment. This includes: placing marker points (such as infrared laser reflection points) on the surface of the physical devices to create corresponding virtual anchor points; achieving millimeter-level initial mapping of physical-virtual space through affine transformation matrices (such as singular value decomposition algorithms) and residual compensation fields; and verifying global accuracy (such as uncalibrated point testing) to ensure that nonlinear distortion is ≤ the system tolerance threshold.

[0074] The head motion tracking module is used to capture head movement in real time and generate pose data. This includes: acquiring angular velocity, acceleration, and magnetic field data through a multi-axis IMU sensor; fusing quaternions (gravity / magnetic north vector) to construct an attitude matrix and separating the actual motion acceleration; generating a displacement vector through double integration and suppressing noise through Kalman filtering to output a synchronized pose matrix.

[0075] The dynamic drift compensation module is used to eliminate coordinate drift caused by head movements. It includes: synthesizing a feedforward mapping matrix to offset coordinate system shifts caused by pose changes; detecting gaze deviation using an eye-tracking device; and dynamically updating the mapping matrix through gradient optimization to converge residual errors.

[0076] The spatial error optimization module is used to correct mapping distortion in real time. This includes: detecting projection deviations based on reference anchor points (such as virtual dashboards); fusing feedforward and feedback mappings to generate the final mapping matrix; and dynamically balancing response speed and stability with weights.

[0077] The behavior scoring module is used to comprehensively evaluate user actions. This includes: converting physical coordinates to virtual coordinates using a final mapping matrix, calculating dynamic spatial residuals, generating credibility weights to quantify mapping reliability, and outputting a comprehensive score that integrates spatial accuracy and operational logic.

[0078] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Since the above embodiments are substantially similar to the method embodiments, their descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0080] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for scoring user operation behavior in a simulation training system, characterized in that, The method includes the following steps: Establish the initial mapping relationship between the physical coordinate system and the virtual coordinate system on the physical device; Monitor user head movement data and construct a pose matrix describing head movement; A feedforward mapping matrix is ​​generated based on the pose matrix and the initial spatial mapping relationship to compensate for coordinate system drift caused by head movement in real time. Residual mapping errors are detected by reference anchor points in a virtual coordinate system, spatial error vectors are calculated, and feedforward mapping matrix is ​​dynamically optimized to generate the final mapping matrix. The physical operation coordinates are converted into virtual space coordinates using the final mapping matrix, and the user's operation behavior is comprehensively scored by combining the credibility weight.

2. The scoring method according to claim 1, characterized in that, Establishing the initial mapping relationship between the physical coordinate system and the virtual coordinate system on the physical device includes: Physical markers are deployed on the surface of physical devices to form a set of physical markers, and a set of virtual anchors corresponding to the physical markers is created in the virtual environment. Calculate the affine transformation matrix that minimizes the error between the physical markers and the virtual anchors, i.e.: In the formula, is the optimal coordinate transformation matrix from the physical coordinate system to the virtual coordinate system; M is the affine transformation matrix to be solved; V is the point cloud matrix of the virtual anchor points; P is the point cloud matrix of the physical marker points; F is the identifier of the Frobenius norm, defined as the square root of the sum of the squares of all elements of the matrix; Next, calculate the residual vector used to compensate for assembly errors, i.e.: In the formula, It is the residual vector; Based on the optimal coordinate transformation matrix and residual vector, an initial mapping relationship between the physical and virtual coordinate systems is established to reduce the mapping error to the hardware limit, namely: In the formula, Set the coordinates in the virtual coordinate system; The coordinates are in the physical coordinate system; T is the transpose operator. Verify the global accuracy of the initial mapping relationship using uncalibrated points.

3. The scoring method according to claim 2, characterized in that, The verification of the global accuracy of the initial spatial mapping relationship through uncalibrated points includes: Select a set of test points on the physical device that were not calibrated, substitute their coordinates into the optimal coordinate transformation matrix, calculate the virtual projection position, and compare the deviation between the actual virtual coordinates and the projected coordinates, i.e.: In the formula, This represents the spatial mapping deviation value for the i-th uncalibrated point; Let be the true theoretical coordinates of the i-th uncalibrated point in the virtual coordinate system; The projected coordinates of the i-th uncalibrated point are calculated using the mapping relationship; The spatial mapping deviation value of the uncalibrated points is compared with the system tolerance threshold. When the spatial mapping deviation value is greater than the system tolerance threshold, a residual compensation field is constructed to eliminate nonlinear distortion, i.e.: In the formula, This is the residual compensation vector at the point to be compensated; The physical coordinates of the point to be compensated; Let i be the coordinates of the i-th physical marker point; The weight coefficient for the i-th physical marker point; Gaussian kernel function; Furthermore, the topology of the device skeleton used for deformation propagation analysis is extracted, and the local deformation of the physical device is dynamically mapped to the precise compensation displacement of the target node in the virtual space, i.e.: In the formula, Let be the deformation compensation vector of the j-th key structural point in virtual space; Let j be the set of neighbor nodes of the j-th key structural point; Let be the deformation transfer coefficient from the i-th critical structural point to the j-th critical structural point; Let be the measured physical deformation direction of the i-th neighbor node; Generate a deformation Jacobian correction matrix for dynamically adjusting coordinate transformation, convert the deformation compensation vector into matrix operation form, and output the adjusted coordinate transformation matrix; and ensure that the global error probability converges within the system tolerance threshold by randomly sampling verification points.

4. The scoring method according to claim 1, characterized in that, The process of monitoring user head movement data and constructing a pose matrix describing head movement includes: The state data includes the velocity angle vector, acceleration vector, and magnetic field strength vector; A reference quaternion is generated based on the acceleration vector and the magnetic field strength vector; Calculate attitude quaternions based on velocity angle vectors; The weighted fusion of reference quaternions and attitude quaternions is used to output a fused attitude quaternion for the final head attitude in spatial localization. The true translational acceleration used to transform local coordinate system acceleration to physical coordinate system acceleration is calculated by fusing attitude quaternions and acceleration vectors. In the formula, This is the original acceleration vector in the physical coordinate system; To be the inverse quaternion of the fusion attitude quaternion; It is the acceleration vector; The true motion acceleration vector, which eliminates the interference of gravity on the Z-axis acceleration, is calculated using the original acceleration vector and the unit gravity vector. In the formula, This represents the actual acceleration vector. It is the unit vector of gravity; The real motion acceleration vector is fused with the historical real motion acceleration vector using a time-varying weighted method, that is: ； In the formula, This is the filtered acceleration output vector; This represents the actual acceleration vector at the current moment. This is the filtered acceleration output vector from the previous moment; For adaptive Kalman gain; The displacement vector is calculated by double integration of the filtered acceleration data, i.e.: In the formula, in the formula, It is the displacement vector; A homogeneous transformation matrix is ​​constructed by combining the displacement vector and the attitude matrix, and a corrected transformation matrix that is synchronized with the actual physical motion is output by linear interpolation. Select reference marker points to perform bidirectional coordinate transformation to verify the transformation fidelity of the correction transformation matrix. If the correction transformation matrix does not meet the transformation fidelity, correct the calculated displacement vector. The pose matrix is ​​constructed based on the attitude matrix and the corrected optimal displacement vector.

5. The scoring method according to claim 4, characterized in that, The weighted fusion reference quaternion and attitude quaternion are obtained in the following way: The projection of the gravity direction from the physical coordinate system onto the local coordinate system is calculated using attitude quaternions, i.e.: In the formula, This is the theoretical direction vector of gravity. This is the standard gravity vector in the physical coordinate system; For attitude quaternions; It is the inverse quaternion of the attitude quaternion; The drift error of the quantized gyroscope attitude in the gravity direction, reflecting the attitude reliability, is calculated using attitude quaternions, the theoretical gravity direction vector, and the gravity unit vector. In the formula, This is the error vector for the direction of gravity. This is the gradient function of the gravity error; The magnitude of the gravity direction error vector, a preset upper limit for static weights, and a dynamic sensitivity coefficient are used to calculate an adaptive balance between static drift suppression and dynamic response preservation, i.e.: In the formula, For fusion weighting coefficients; This represents the upper limit of static weights. This refers to the dynamic sensitivity coefficient. Let be the magnitude of the gravity direction error vector; The formula for calculating the fused attitude quaternion is: In the formula, To fuse pose quaternions; For reference quaternions; It is a quaternion of attitude.

6. The scoring method according to claim 4, characterized in that, The step of selecting reference marker points to perform bidirectional coordinate transformation to verify the transformation fidelity of the correction transformation matrix includes: A bidirectional transformation is performed on the reference marker point. The forward transformation maps the physical coordinates of the reference marker point to the local coordinate system of the current head, and the inverse transformation remaps the result back to the physical coordinate system. The bidirectional coordinate mapping residual is then calculated. In the formula, Use as a reference marker point; This is a forward transformation operation; This is the inverse transformation operation; For coordinate bidirectional mapping residuals; The coordinate bidirectional mapping residual is compared with a preset residual threshold. If the coordinate bidirectional mapping residual is greater than the residual threshold, the displacement vector is corrected during calculation. In the formula, This is the corrected optimal displacement vector; Let represent the three-dimensional displacement vector to be solved, where Represents the set of real numbers in three-dimensional space; IMU confidence weights; The displacement vector calculated by the IMU is: The displacement vector for optical positioning is: In the formula, Let be the three-dimensional coordinates of the i-th marker point in the local coordinate system; Indicates the actual observed value; For projection functions; Let be the homogeneous transformation matrix containing the three-dimensional displacement vector to be solved and a fixed attitude matrix; where This is the attitude matrix.

7. The scoring method according to claim 1, characterized in that, The method for calculating the spatial error vector and dynamically optimizing the feedforward mapping matrix is ​​as follows: Define the theoretical gaze area of ​​a fixed reference object; The user gaze vector output by the eye-tracking device is projected onto the virtual space through a feedforward mapping matrix. The user's initial gaze coordinates in the virtual space are obtained, and transient noise is filtered to output the effective gaze coordinates after clear denoising of biomechanical features. Calculate the projection deviation between the effective gaze point coordinates after denoising and the center coordinates of the reference anchor point, i.e.: In the formula, This is the spatial error vector; The center coordinates of the reference anchor point; These are the effective gaze point coordinates after denoising. The feedforward mapping matrix is ​​updated by moving the spatial error vector along the direction of error reduction, i.e.: In the formula, This is the optimized feedback mapping matrix; For matrix gradient operators; For adaptive step size.

8. The scoring method according to claim 7, characterized in that, The method for generating the final mapping matrix is ​​as follows: The dynamic weight coefficients of the feedback mapping matrix and feedforward mapping matrix after dynamic fusion optimization are calculated using the spatial error vector, i.e.: In the formula, These are dynamic weighting coefficients; The characteristic scale constant; The formula for calculating the final mapping matrix is: In the formula, This is the final mapping matrix; This is the feedforward mapping matrix.

9. The scoring method according to claim 1, characterized in that, The comprehensive scoring of user behavior based on credibility weights includes: The final mapping matrix is ​​used to perform an instantaneous matrix transformation calculation on the physical markers in the physical marker set, that is: In the formula, Let i be the coordinates of the i-th physical marker point; The corrected virtual coordinates of the i-th physical marker point in the virtual coordinate system; The dynamic spatial residual used to quantify spatial mapping errors is calculated by correcting the virtual coordinates and the original measurements from the VR system. For dynamic space residuals; These are the original measurement values ​​from the VR system; Based on the dynamic spatial residual and a preset error tolerance threshold, physical spatial distortion is converted into scoring confidence, i.e.: In the formula, As a credibility weight; This is the preset error tolerance threshold; d is the preset error failure threshold; d is the attenuation slope coefficient. By adding a credibility weight to the existing scoring model, a comprehensive quantitative evaluation result of user behavior is output, namely: In the formula, Score the final behavior; The original spatial score generated based on spatial coordinate data; Logical scoring.

10. A system for implementing the scoring method according to any one of claims 1-9, characterized in that, include: The initial spatial mapping module is used to establish the coordinate system alignment between physical devices and the virtual environment; The head motion tracking module is used to capture head movement in real time and generate pose data; The dynamic drift compensation module is used to eliminate coordinate drift caused by head movement; Spatial error optimization module, used to correct mapping distortion in real time; The behavior scoring module is used to comprehensively evaluate user actions.

Images