Shooting simulation training method and system based on virtual reality

Abstract

The invention relates to the technical field of virtual reality simulation training, in particular to a shooting simulation training method and system based on virtual reality, and the method comprises the steps that a virtual training space is constructed, and dynamic light and shadow and terrain disturbance variables are injected into a visual environment model; loading a physical attribute template by the trajectory simulation engine; and loading a standard attitude sequence into the instrument action library. In training, the system captures real-time posture, instrument operation and environment interaction data of a trainee, inputs the data into a trajectory simulation engine to generate a trajectory prediction data set containing a speed curve and a deflection angle, compares the trajectory prediction data set with a standard posture frame by frame, and generates an action analysis report. According to the method, the authenticity and adaptability of training are enhanced by simulating continuously changing environmental physical interference; through full-process trajectory simulation and action traceability, accurate evaluation and deviation correction guidance of shooting operation details are realized.

Description

Technical Field

[0001] This invention relates to the field of virtual reality simulation training technology, and in particular to a shooting simulation training method and system based on virtual reality. Background Technology

[0002] Currently, virtual reality-based shooting simulation training systems are in use. These systems typically construct virtual scenes and equipment models, enabling interaction between the virtual environment and the simulated equipment through motion capture devices. Their core technical logic is to capture the trainee's basic movements and aiming points, and determine the hit result based on simple geometric ray collision detection or a preset trajectory. System evaluation focuses primarily on macro-level indicators such as final hit rate and reaction time. This technical approach constitutes the mainstream mode of existing virtual shooting training.

[0003] Existing technical solutions have limitations in terms of the realism of environmental simulation and the depth of training evaluation. Most systems employ static training environments or those with only simple rule variations, lacking simulation of continuous, random physical disturbances in the natural environment. Training occurs under idealized, constant conditions, which makes simulation training insufficient in improving trainees' ability to cope with complex real-world environments. Furthermore, the system's evaluation and feedback mechanisms are superficial, typically only relating to the start and end points of the action, failing to continuously model and finely analyze the complete physical process from the generation of firing intent and equipment operation to the projectile's trajectory. Training data cannot effectively reveal the causal relationship between minor flaws in the action and the final ballistic result, causing evaluation feedback to remain at the outcome level, making it difficult to support the diagnosis and correction of fundamental defects in the technical actions.

[0004] There is a need for a virtual reality shooting simulation training method that can simulate continuous dynamic physical interference in a real environment and achieve continuous and refined physical simulation and motion source analysis of the entire shooting process under the interaction of "human-equipment-environment". Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and to propose a shooting simulation training method and system based on virtual reality.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a shooting simulation training method based on virtual reality, comprising: Construct a virtual training space that includes a visual environment model, a ballistic simulation engine, and a library of weapon actions; Dynamic lighting and terrain perturbation variables are injected into the visual environment model; Activate the ballistic simulation engine and load the physical attribute templates corresponding to the virtual firing equipment; Start the equipment action library and load the standard shooting posture sequence; Perform environment initialization calibration to align the rendering output of the visual environment model, the initial state of the ballistic simulation engine, and the baseline data of the instrument action library. Capture the real-time gaze focus coordinates and body posture coordinates of the trainee in the visual environment model; Capture the sequence of grip angles and trigger pressure during the operation of a virtual shooting device; The real-time gaze focus coordinates, body posture coordinates, instrument grip angle sequence, and trigger pressure sequence are input into the ballistic simulation engine; The ballistic simulation engine generates a ballistic prediction dataset containing virtual projectile velocity curves and virtual projectile deflection angles based on physical property templates. The ballistic prediction dataset is compared frame by frame with the standard firing posture sequence in the weapon action library; Based on the comparison results, a shooting action analysis report is generated, which includes action deviation markers and trajectory consistency ratings.

[0007] As a further aspect of the present invention, the step of constructing a virtual training space comprising a visual environment model, a ballistic simulation engine, and a weapon action library specifically includes: The three-dimensional scene mesh data and material mapping relationship of the virtual training space are generated using computer-aided design tools. Mark potential interaction areas and visually obstructed areas in the 3D scene mesh data; Bind the illumination intensity variation curve of the dynamic lighting variable and the surface height variation curve of the terrain disturbance variable to the corresponding vertices of the 3D scene mesh data; Configure virtual air density parameters, virtual gravity parameters, and virtual material penetration parameters for the ballistic simulation engine based on the physical property template; Based on the standard shooting posture sequence, posture constraint rules are constructed for the equipment action library, including elbow joint angle range, shoulder joint angle range, and spinal deviation tolerance.

[0008] As a further aspect of the present invention, the step of performing environment initialization calibration specifically includes: Send a calibration command to the visual environment model, triggering the visual environment model to output a calibration screen containing a specific color block array and scale indicators; Read the chromaticity values ​​of a specific color block array and the pixel dimensions of the scale marker in the calibration image; By comparing the chromaticity values ​​with the preset standard chromaticity threshold, color calibration parameters for the visual environment model are generated. By comparing pixel size with a preset standard scale threshold, size calibration parameters for the visual environment model are generated. Synchronize the color calibration parameters and size calibration parameters to the ballistic simulation engine and the instrument motion library; The ballistic simulation engine adjusts the virtual distance calculation scale in the physical property template based on the size calibration parameters; The instrument action library adjusts the posture judgment thresholds related to visual aiming in the standard shooting posture sequence based on color calibration parameters.

[0009] As a further aspect of the present invention, the step of capturing the real-time gaze focus coordinates and body posture coordinates of the trainee in the visual environment model specifically includes: Analyze the head orientation quaternion data and eye-tracking image data transmitted by the virtual reality headset; Convert the head direction quaternion data into an Euler angle dataset that includes pitch, yaw, and roll angles; Pupil contour recognition and corneal reflection point localization are performed on eye-tracking image data, and the coordinates of the real-time gaze focus in the three-dimensional coordinate system of the visual environment model are calculated by combining the Euler angle dataset. Analyzing the joint position data stream transmitted by the virtual reality body tracking kit; Based on the spatial positions of the shoulder, hip, and foot joints in the joint position data stream, body posture coordinates are obtained through inverse kinematics calculations. The body posture coordinates include the trunk tilt angle and the coordinates of the center of gravity projection point.

[0010] As a further aspect of the present invention, the step of capturing the sequence of instrument grip angles and trigger pressure sequences during the operation of the virtual shooting instrument specifically includes: Receives inertial measurement unit data and capacitive sensing array data transmitted from the virtual reality controller; Extract the real-time orientation matrix and angular velocity vector of the controller in the virtual training space from the inertial measurement unit data; The grip angle between the muzzle pointing axis and the standard pointing axis of the virtual shooting device is calculated based on the real-time orientation matrix and recorded in chronological order to form a sequence of grip angles. The capacitance changes of multiple sensing units covering the virtual trigger area are analyzed from the capacitive sensing array data. The changes in capacitance values ​​of multiple sensing units are weighted and fused to map them into continuous trigger pressure values, which are then recorded in chronological order to form a trigger pressure sequence.

[0011] As a further aspect of the present invention, the step of the ballistic simulation engine generating a ballistic prediction dataset containing virtual projectile velocity curves and virtual projectile deflection angles based on a physical property template specifically includes: The real-time gaze focus coordinates and body attitude coordinates are input into the initial conditions module of the ballistic simulation engine. The initial conditions module calculates the launch position vector and initial direction vector of the virtual projectile. The instrument grip angle sequence and trigger pressure sequence are input into the perturbation calculation module of the ballistic simulation engine. The perturbation calculation module, combined with the virtual air density parameter in the physical property template, calculates the initial directional perturbation vector caused by the unstable instrument grip and the initial velocity fluctuation caused by the unstable trigger pressure. The core solver of the ballistic simulation engine takes the launch position vector, initial direction vector, initial direction perturbation vector and initial velocity fluctuation as input, and combines the virtual gravity parameters and virtual material penetration parameters in the physical property template to calculate the trajectory of the virtual projectile in the virtual training space through numerical integration iteration. Extract the velocity magnitude and direction corresponding to each time step from the motion trajectory to form a virtual projectile velocity curve; The difference between the actual impact point coordinates of the virtual projectile and the theoretical aiming point coordinates is calculated to obtain the deflection angle of the virtual projectile.

[0012] As a further aspect of the present invention, the step of comparing the ballistic prediction dataset with the standard firing posture sequence in the weapon action library frame by frame specifically includes: Synchronize and align the timeline of the ballistic prediction dataset with the timeline of the standard firing attitude sequence; For each synchronized time frame, extract the virtual projectile velocity value and virtual projectile deflection angle value corresponding to the time frame from the ballistic prediction dataset; Extract the standard velocity range, standard deflection tolerance interval, standard elbow joint angle, and standard shoulder joint angle corresponding to the time frame from the standard shooting posture sequence; Determine whether the virtual projectile's velocity value falls within the standard velocity range and mark it with a velocity consistency indicator; Determine whether the virtual projectile deflection angle exceeds the standard deflection tolerance range and mark the deviation as abnormal; By combining the sequence of grip angles of the device, calculate the angle difference between the actual elbow joint angle and the actual shoulder joint angle of the current frame and the standard value, and mark the posture deviation.

[0013] As a further aspect of the present invention, the step of generating a shooting action analysis report containing action deviation markers and trajectory consistency ratings based on the comparison results specifically includes: Calculate the proportion of time frames in which the velocity consistency flag is true, and generate the trajectory velocity consistency rate. Calculate the proportion of deviation anomalies marked as true across all time frames to generate ballistic deviation frequency; The proportion of attitude deviation indicators exceeding a preset threshold in all time frames is counted to generate an attitude instability index. The trajectory velocity consistency rate, ballistic deviation frequency, and attitude instability index are input into a predefined rating mapping table. The rating mapping table outputs a trajectory consistency rating, which includes excellent, qualified, and need-to-improve levels. Based on the changing patterns of the instrument grip angle sequence and trigger pressure sequence, the grip shake pattern, trigger snap pattern, or aiming lag pattern are identified, and the identified patterns are added to the shooting action analysis report as action deviation markers.

[0014] As a further aspect of the present invention, the method further includes a dynamic environment response step in the virtual training space: The visual environment model calculates collision events between virtual projectiles and scene objects in real time based on the trajectory of virtual projectiles in the ballistic prediction dataset. When a collision event is triggered, the visual environment model generates a bullet hole indentation geometric deformation and a corresponding fragment particle effect on the 3D scene mesh data based on the virtual material penetration parameters in the physical property template. Meanwhile, the visual environment model sends the location coordinates and collision intensity values ​​of the collision event to the ballistic simulation engine; The ballistic simulation engine updates the physical property template of subsequent virtual projectiles based on the collision intensity value. The update includes adjusting the virtual air density parameter to simulate the effect of dust, or adjusting the virtual gravity parameter to simulate shock wave disturbance.

[0015] As a further aspect of the present invention, the present invention also includes a shooting simulation training system based on virtual reality, the system including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, it implements the steps of the shooting simulation training method based on virtual reality as described above.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: By injecting dynamic lighting and terrain disturbance variables into the visual environment model, the system can generate non-preset, continuously changing environmental conditions. Dynamic lighting variables directly alter the light intensity, angle, and shadow distribution in the virtual scene, continuously interfering with the trainee's visual recognition and aiming. Terrain disturbance variables change the physical properties of the trainee's virtual foothold, affecting the stability of their body posture. This technical solution forces the trainee's perception and motion control system to process and compensate for environmental inputs in real time, breaking the operational patterns formed in environments with fixed parameters. The training process thus encompasses cultivating the ability to maintain operational stability under uncertain environmental factors, improving the realism of the simulation environment and complex real-world scenes, and enhancing the training transfer value.

[0017] The ballistic simulation engine generates a ballistic prediction dataset containing velocity curves and deflection angles based on physical property templates, and compares it frame-by-frame with a standard firing posture sequence. This approach establishes a high-fidelity simulation link from the mechanical input of the instrument operation to the physical output of the projectile's flight. The physical property templates ensure that the dynamic characteristics of different virtual instruments are accurately mapped to the ballistic generation. The generated ballistic data not only points to the final impact point but also completely records the changes in the projectile's motion state after leaving the barrel. By comparing it frame-by-frame with the standard action, the system can correlate the final deviation of the trajectory with the posture data at specific time points in the action execution process. This allows the analysis of firing accuracy to trace back to instantaneous anomalies in specific operational parameters such as grip angle and trigger pressure curve, enabling a deeper understanding of operational technique flaws from macroscopic results to microscopic process parameters, providing a basis for refined correction of technical actions. Attached Figure Description

[0018] Figure 1 This is a flowchart of the shooting simulation training method based on virtual reality described in this invention; Figure 2 Flowchart for constructing a virtual training space; Figure 3 A flowchart for the environmental initialization calibration process; Figure 4 A time sequence diagram of raw data for equipment operation in virtual reality shooting simulation training; Figure 5 A bar chart comparing the performance of each component in the virtual training space. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] See Figure 1A virtual training space was constructed, comprising a visual environment model, a ballistic simulation engine, and a weapon motion library. Dynamic lighting and terrain disturbance variables to simulate changes in real-world conditions were pre-injected into the visual environment model. During training, the ballistic simulation engine and weapon motion library were activated simultaneously. The ballistic simulation engine loaded a physical attribute template matching the current virtual shooting weapon, while the weapon motion library loaded a preset standard shooting posture sequence. An environment initialization calibration was performed to unify the rendering output of the visual environment model, the initial state of the ballistic simulation engine, and the baseline data of the weapon motion library. During training, two levels of data were captured in real time: first, the trainee's real-time gaze focus coordinates and body posture coordinates in the visual environment model; second, the weapon grip angle sequence and trigger pressure sequence generated during the operation of the virtual shooting weapon. The captured real-time gaze focus coordinates, body posture coordinates, weapon grip angle sequence, and trigger pressure sequence were input into the ballistic simulation engine. The ballistic simulation engine called the loaded physical attribute templates to generate a ballistic prediction dataset containing the virtual projectile velocity curve and virtual projectile deflection angle. The generated ballistic prediction dataset is compared frame by frame with the standard shooting posture sequence in the instrument action library. Based on the results of the frame-by-frame comparison, a shooting action analysis report containing action deviation markers and trajectory consistency ratings is finally generated.

[0022] In one embodiment of the present invention, see [reference] Figure 2 Computer-aided design tools generate 3D scene mesh data and material mapping relationships for the virtual training space. Potential interaction areas and visual obstacle areas are marked in the 3D scene mesh data. Potential interaction areas include the locations of fireable targets, and visual obstacle areas include virtual bunkers and buildings. The illumination intensity variation curve of dynamic lighting variables and the surface height variation curve of terrain disturbance variables are bound to the corresponding vertices of the 3D scene mesh data. This binding operation allows the lighting in the virtual training space to simulate day and night cycles, and the terrain to simulate undulations caused by earthquakes or explosions. In some embodiments, the ballistic simulation engine configures virtual air density parameters, virtual gravity parameters, and virtual material penetration parameters based on physical property templates. The virtual air density parameter simulates the environmental differences between mountains and plains, the virtual gravity parameter maintains the Earth's standard value or is adjusted to simulate training on different planets, and the virtual material penetration parameter defines the projectile's ability to penetrate objects of different materials. The equipment motion library is constructed based on the standard shooting posture sequence, including posture constraint rules that include the elbow joint angle range, the shoulder joint angle range, and the spinal deviation tolerance. The elbow joint angle range is limited to 30 degrees to 150 degrees, the shoulder joint angle range is limited to -20 degrees to 90 degrees, and the spinal deviation tolerance is set to no more than 5 degrees.

[0023] In the dynamic environment response step of the virtual training space, the visual environment model calculates collision events between virtual projectiles and scene objects in real time based on the trajectory of virtual projectiles in the ballistic prediction dataset. The calculation of collision events is based on geometric intersection detection of 3D scene mesh data. When a collision event is triggered, the visual environment model generates bullet hole indentation geometry and corresponding fragment particle effects on the 3D scene mesh data according to the virtual material penetration parameters in the physical property template. The generation of bullet hole indentation geometry is achieved through vertex displacement, and the fragment particle effect is simulated by a particle system to simulate flying debris. At the same time, the visual environment model sends the position coordinates and collision intensity value of the collision event to the ballistic simulation engine. The collision intensity value is jointly determined by the virtual projectile velocity and the virtual material penetration parameters. The ballistic simulation engine updates the physical property template of subsequent virtual projectiles based on the collision intensity value. The update operations include adjusting the virtual air density parameter to simulate dust impact or adjusting the virtual gravity parameter to simulate shock wave disturbance. When simulating dust impact, the virtual air density parameter increases with the collision intensity value, and when simulating shock wave disturbance, the virtual gravity parameter changes temporarily in a local area.

[0024] In practical implementation, the binding of dynamic lighting variables and terrain perturbation variables is achieved through the vertex shader program. Light intensity variation curves and ground height variation curves serve as input textures or arrays to drive vertex attributes. Optionally, the material mapping relationship of the 3D scene mesh data includes diffuse maps, normal maps, and specular maps to enhance the realism of the visual environment model. In some embodiments, the marking of potential interaction areas and visual obstacle areas uses an additional semantic layer, which is used as a mask in collision detection and path planning. It can be understood that the virtual material penetration parameter is a multi-dimensional vector, with vector elements corresponding to different material types such as concrete, wood, and metal. The depth and shape of the geometric deformation of the bullet hole indentation are calculated based on the numerical values ​​of the corresponding material in the vector.

[0025] In practice, the generation of the geometric deformation of the bullet hole indentation is described by a formula:

[0026] in: This indicates the depth offset of the bullet hole indentation. Indicates the collision intensity value. This represents the scalar value of the corresponding collision material in the virtual material penetration parameters. This represents the magnitude of the virtual projectile's velocity at the moment of impact. and This is an adjustment coefficient used to control the sensitivity of deformation amplitude and velocity effects. The formula ensures consistency between the geometric deformation and physical properties of bullet hole indentation; high-speed projectiles impacting hard materials produce deep and concentrated indentations, while low-speed projectiles impacting soft materials produce shallow and diffused indentations. Optionally, the number of fragmentation particles is also derived from the variables in the formula, and the number of fragmentation particles is proportional to... The initial velocity of the fragmented particle effect is proportional to... .

[0027] In practical implementation, the data comparison of the dynamic environment response steps is reflected in the difference in the virtual training space state before and after the collision event processing. Before the collision event processing, the 3D scene mesh data maintains its original geometry. After the collision event processing, the 3D scene mesh data undergoes permanent vertex displacement at the collision point, accompanied by the emission of fragmented particle effects lasting for several seconds. The visual environment model sends the location coordinates and collision intensity value of the collision event to the ballistic simulation engine. When the ballistic simulation engine adjusts the virtual air density parameter based on the collision intensity value, the virtual air density parameter increases in a Gaussian distribution around the collision point, with the increase linearly correlated with the collision intensity value. When adjusting the virtual gravity parameter, the virtual gravity parameter temporarily decreases in the area above the collision point to simulate the updraft of the shock wave. It can be understood that these update operations cause the ballistic prediction dataset of subsequent virtual projectiles to include environmental interaction effects. The velocity curve of subsequent virtual projectiles shows attenuation when passing through the dust area, and the deflection angle of subsequent virtual projectiles exhibits random fluctuations within the shock wave disturbance area.

[0028] In one embodiment of the present invention, see [reference] Figure 3 The process involves sending calibration commands to the visual environment model, triggering it to output a calibration screen containing a specific color block array and scale markers. The specific color block array uses a color chart pattern containing 24 standard colors, and the scale markers use a checkerboard pattern with known physical dimensions. The chromaticity values ​​of the specific color block array and the pixel dimensions of the scale markers in the calibration screen are read. The chromaticity values ​​are captured by the image sensor and converted to the CIELab color space. The pixel dimensions are calculated by sub-pixel localization of the checkerboard corner points using an image processing algorithm. The chromaticity values ​​are compared with a preset standard chromaticity threshold to generate color calibration parameters for the visual environment model. The preset standard chromaticity threshold is the theoretical Lab value of the 24 standard colors under standard lighting. The color calibration parameters are a set of three-dimensional lookup tables used to adjust the output colors of the visual environment model's rendering pipeline. Finally, the pixel dimensions are compared with a preset standard scale threshold to generate size calibration parameters for the visual environment model. The preset standard scale threshold is the actual physical size of the checkerboard pattern set in the virtual training space. The size calibration parameters are a scaling factor used to correct the proportional relationship between the virtual world and the perceived world.

[0029] The generated color calibration parameters and size calibration parameters are synchronized to the ballistic simulation engine and the instrument motion library. This synchronization is achieved through inter-process communication. The ballistic simulation engine adjusts the virtual distance calculation ratio in the physical attribute template based on the received size calibration parameters, by multiplying the virtual distance calculation ratio by the reciprocal of the size calibration parameter. The instrument motion library adjusts the posture judgment threshold related to visual aiming in the standard shooting posture sequence based on the received color calibration parameters, by multiplying the posture judgment threshold by a visual difference coefficient derived from the color calibration parameters. In some embodiments, the calibration command is automatically issued by the system after the virtual training space is loaded, and the calibration screen is displayed within the display area of ​​the virtual reality headset. It is understood that the preset standard color threshold and preset standard scale threshold are stored in separate calibration configuration files, which can be changed according to different training scenarios and hardware models.

[0030] In practice, generating color calibration parameters involves calculating and compensating for differences in chromaticity values. Optionally, the generation of color calibration parameters can be described using a formula:

[0031] in: This represents the color difference of a single color block. This indicates the chromaticity value read from the calibration screen. This indicates a preset standard chromaticity threshold; the final generated color calibration parameters include those used to... Mapped to The interpolation function coefficients. Size calibration parameters are generated by calculating the ratio of the pixel size to a preset standard scale threshold. This is equal to a preset standard scale threshold divided by the virtual length unit corresponding to the pixel size. In some embodiments, the preset standard scale threshold is set to a side length of 1 virtual meter for each square in the checkerboard pattern, and the measured side length corresponding to the pixel size is N pixels. The virtual length corresponding to N pixels is calculated by back-calculating the size calibration parameters using a virtual camera model with known intrinsic parameters. This is the ratio of 1 virtual meter to the calculated virtual length.

[0032] In practical implementation, data comparison is reflected in the changes in system parameters before and after calibration. Before calibration, the visual environment model may have deviations in chromaticity values ​​due to differences in display devices. Before calibration, the pixel size of the scale markers may deviate from the theoretical value due to optical distortion of the head-mounted device or different user wearing positions. The color calibration parameters generated after calibration are a set of correction values, reducing the chromaticity deviation from an average ΔC=15 to ΔC<2. The size calibration parameters generated after calibration are a proportional value, reducing the scale error from the initial 5% to within 1%. The ballistic simulation engine adjusts the virtual distance calculation ratio based on the size calibration parameters. Before adjustment, the virtual distance calculation ratio defaulted to 1:1; after adjustment, the virtual distance calculation ratio becomes 1: This ensures that the virtual projectile impact point calculated in the ballistic prediction dataset matches the actual perceived distance. The device motion library adjusts the attitude judgment threshold based on color calibration parameters. Before adjustment, the attitude judgment threshold related to visual aiming is a fixed value; after adjustment, the attitude judgment threshold is dynamically scaled according to the calibrated visual difference coefficient. For example, when the color calibration parameters show excessive saturation in the red channel, the visual difference coefficient increases, and the attitude judgment threshold is correspondingly widened to compensate for the increased difficulty of visual judgment. It can be understood that environmental initialization calibration ensures that the subsequently captured real-time gaze focus coordinates and body posture coordinates are established on standardized visual and spatial benchmarks. Optionally, the calibration process is performed at the beginning of each training session to eliminate systematic errors caused by differences in device status and user physiology.

[0033] In one embodiment of the present invention, head orientation quaternion data and eye-tracking image data transmitted by a virtual reality headset are analyzed. The head orientation quaternion data is acquired from an inertial measurement unit at a rate of 90 frames per second, and the eye-tracking image data is captured by a near-infrared camera integrated inside the headset at a rate of 120 frames per second. The head orientation quaternion data is converted into an Euler angle dataset containing pitch, yaw, and roll angles. The conversion process uses a standard quaternion-to-Euler angle conversion formula, with the pitch angle ranging from ±90 degrees, the yaw angle ranging from 0 to 360 degrees, and the roll angle ranging from ±180 degrees. Pupil contour recognition and corneal reflector localization are performed on the eye-tracking image data. The coordinates of the real-time gaze focus in the three-dimensional coordinate system of the visual environment model are calculated using a geometric optics model based on the calculated Euler angle dataset. Pupil contour recognition uses an ellipse fitting algorithm, and corneal reflector localization is achieved by identifying the Purkinje spot formed on the cornea by a near-infrared light source.

[0034] The joint position data stream transmitted by the virtual reality body tracking kit is analyzed. This data stream contains the three-dimensional coordinates of at least 15 joints in the virtual training space, with a data update frequency of 60 times per second. Based on the spatial positions of the shoulder, hip, and foot joints in the joint position data stream, body posture coordinates are calculated through inverse kinematics. These body posture coordinates include the trunk tilt angle and the coordinates of the center of gravity projection point. The trunk tilt angle is obtained by calculating the angle between the vector connecting the midpoints of the shoulder and hip joints and the vertical axis. The coordinates of the center of gravity projection point are calculated using a weighted centroid model based on joint positions and projected onto the horizontal plane. In some embodiments, the virtual reality headset and the virtual reality body tracking kit communicate with the main computing unit via a wireless protocol. A timestamp synchronization mechanism ensures that the head orientation quaternion data, eye-tracking image data, and joint position data stream have a unified time reference. It is understood that the calculation of real-time gaze focus coordinates relies on an accurate gaze estimation model, the model parameters of which are obtained through a user calibration process.

[0035] In practice, the geometrical optical model for calculating the coordinates of the real-time gaze focus is described by a single formula:

[0036] in: This represents the coordinate vector of the final real-time gaze focus in the three-dimensional coordinate system of the visual environment model. This represents the position vector of the optical center of a virtual reality headset in a three-dimensional coordinate system. This represents the head rotation matrix composed of Euler angle datasets. This represents a scalar measure indicating the estimated distance from the optical center to the focal point of gaze. This represents the normalized coordinate vector of the pupil center on the camera image plane. This represents the normalized coordinate vector of the corneal reflection point on the camera image plane. The formula maps two-dimensional eye-tracking features to three-dimensional virtual space through vector operations. Optionally, a distance scalar is estimated. Based on the known virtual training space scale and the approximate anatomical parameters of the user's eyes, or dynamically calculated from binocular images using stereoscopic vision principles in advanced mode.

[0037] In practical implementation, data comparison is reflected in the mapping relationship between the original sensor data and the processed coordinate data. The original head orientation quaternion data is an array of four floating-point numbers, while the converted Euler angle dataset consists of three floating-point numbers representing pitch, yaw, and roll, respectively. The original eye-tracking image data is a grayscale image frame. After pupil contour recognition and corneal reflection point localization, two-dimensional pixel coordinates are obtained. Finally, the real-time gaze focus coordinates calculated by combining the head rotation matrix are floating-point coordinates in three-dimensional space. In the joint position data stream, the original coordinates of the shoulder, hip, and foot joints are discrete three-dimensional points. In the body posture coordinates calculated through inverse kinematics, the trunk tilt angle is a scalar angle value between 0 and 180 degrees, and the center of gravity projection point coordinates are two-dimensional planar coordinates. When calculating the body posture coordinates, the spatial positions of the shoulder and hip joints are used to define the trunk axis, and the spatial positions of the foot joints are used to constrain the support polygon. The center of gravity projection point coordinates falling within this support polygon indicate a stable posture, while falling outside the support polygon indicates a posture imbalance. It is understood that real-time gaze focus coordinates and body posture coordinates together form the spatial basis for evaluating the trainee's aiming posture and stability. In some embodiments, the virtual reality body tracking kit employs a fusion scheme based on inertial measurement units and ultra-wideband positioning, with joint position data streams subjected to sensor fusion filtering before transmission to reduce noise. Optionally, inverse kinematics calculations introduce skeletal length constraints and joint range of motion constraints to ensure that the calculated body posture coordinates conform to the human biomechanical model.

[0038] In one embodiment of the present invention, inertial measurement unit (IMU) data and capacitive sensing array data transmitted by a virtual reality controller are received. The IMU data includes raw readings from a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. The capacitive sensing array data comes from multiple capacitive sensing units arranged in the controller grip and trigger area. The real-time orientation matrix and angular velocity vector of the controller in the virtual training space are extracted from the IMU data. The real-time orientation matrix is ​​obtained by fusing the accelerometer, gyroscope, and magnetometer data using a sensor fusion algorithm. The angular velocity vector is directly derived from the three-axis angular velocity values ​​measured by the gyroscope. Based on the real-time orientation matrix, the grip angle between the muzzle pointing axis of the virtual shooting device and the system-defined standard pointing axis is calculated. The grip angle is obtained by calculating the angle between the two vectors and is recorded sequentially at fixed sampling time intervals to form a grip angle sequence. The capacitance changes of multiple sensing units covering the virtual trigger area are analyzed from the capacitive sensing array data. These capacitance changes reflect changes in finger contact area and pressure. The capacitance changes of multiple sensing units are weighted and fused. This weighting and fusion uses weighting coefficients assigned based on the spatial location of the sensing units, mapping them to continuous trigger pressure values, and recording them in the same temporal order to form a trigger pressure sequence. In some embodiments, the communication between the virtual reality controller and the main system employs a low-latency wireless protocol, with a data transmission rate exceeding 1000 times per second to capture subtle operational dynamics. See Table 1.

[0039] Table 1: Sequence Table of Instrument Grip Angles

[0040] The process by which the ballistic simulation engine generates a ballistic prediction dataset based on a physical property template is as follows: Real-time gaze focus coordinates and body posture coordinates are input into the initial conditions module of the ballistic simulation engine. This module calculates the launch position vector and initial direction vector of the virtual projectile. The launch position vector is determined based on the muzzle position of the 3D model of the virtual firing device, and the initial direction vector is obtained by normalizing the difference vector between the real-time gaze focus coordinates and the launch position vector. The device grip angle sequence and trigger pressure sequence are input into the perturbation calculation module of the ballistic simulation engine. This module, combined with the virtual air density parameter in the physical property template, calculates the initial direction perturbation vector caused by unstable device grip and the initial velocity fluctuation caused by unstable trigger pressure. The core solver of the ballistic simulation engine takes the launch position vector, initial direction vector, initial direction perturbation vector, and initial velocity fluctuation as input, and, combined with the virtual gravity parameter and virtual material penetration parameter in the physical property template, calculates the trajectory of the virtual projectile in the virtual training space through numerical integration iteration. The velocity magnitude and direction corresponding to each time step are extracted from the calculated trajectory to form a virtual projectile velocity curve. The difference between the actual impact point coordinates and the theoretical aiming point coordinates of the virtual projectile is calculated to obtain the virtual projectile deflection angle. Optionally, the fourth-order Runge-Kutta method is used for numerical integration to ensure computational accuracy and stability, and the time step is dynamically adjusted according to the complexity of the virtual training space.

[0041] In practice, the disturbance calculation module uses a formula to describe the initial velocity fluctuation:

[0042] in: This represents a scalar quantity representing the initial velocity fluctuation. This represents the disturbance sensitivity coefficient, which is jointly determined by the virtual air density parameter in the physical property template and the characteristics of the virtual firing device. Indicates the trigger pressure sequence in time The first derivative at that point is the rate of change of pressure, and the integration interval is... This represents the complete time window for trigger pull. The formula quantifies the impact of trigger operation smoothness on initial velocity; the more drastic the change in trigger pressure, the greater the fluctuation in initial velocity. The calculation of the initial orientation perturbation vector is directly linearly related to the instantaneous values ​​and rate of change of the instrument grip angle sequence. In some embodiments, data comparison is reflected in the conversion between raw sensor data and higher-order physical quantities. The raw inertial measurement unit data consists of nine-axis sensor readings, which, after processing, become an instrument grip angle sequence describing the spatial orientation of the instrument. The raw capacitive sensing array data consists of digital capacitance values ​​from multiple channels, which, after weighted fusion, become a trigger pressure sequence reflecting finger pressure. The input to the core solver is elevated from low-level operational data to high-level physical vectors characterizing launch conditions, and its output ballistic prediction dataset further includes ballistic parameters such as virtual projectile velocity curves and virtual projectile deflection angles.

[0043] See Figure 4 This is a time-series graph of raw equipment operation data from virtual reality shooting simulation training, showing the changes in two key operational indicators from 1000ms to 1100ms. During the rapid increase in trigger pressure between 1025 and 1040ms, the grip angle fluctuates significantly, likely due to muscle tension in the hand leading to decreased weapon grip stability when the trainee pulls the trigger forcefully. At 1035ms, the grip angle reaches its maximum value (2.5 degrees), coinciding with the rapid increase in trigger pressure. This indicates that grip wobbling at this moment may have a significant negative impact on ballistic stability and requires close attention in subsequent training. This graph is a core basis for motion quality assessment in the virtual reality shooting training system and can be directly used to generate "motion deviation markers," such as marking 1035ms as "grip wobbling mode." Combined with ballistic simulation data, the specific impact of grip wobbling and trigger pressure fluctuations on the bullet's initial velocity and deflection angle can be further analyzed, providing trainees with precise motion correction suggestions.

[0044] In one embodiment of the present invention, the time axis of the ballistic prediction dataset is synchronized and aligned with the time axis of the standard firing posture sequence. The synchronization and alignment are based on a unified world clock timestamp. Each data point in the ballistic prediction dataset is correlated with each posture frame in the standard firing posture sequence using a nearest neighbor interpolation method. For each synchronized time frame, the virtual projectile velocity value and virtual projectile deflection angle value corresponding to the time frame are extracted from the ballistic prediction dataset. The unit of the virtual projectile velocity value is meters per second, and the unit of the virtual projectile deflection angle value is angle minutes. The standard velocity range, standard deflection tolerance interval, standard elbow joint angle, and standard shoulder joint angle corresponding to the same time frame are extracted from the standard firing posture sequence. The standard velocity range is a closed interval, the standard deflection tolerance interval is an open interval, and the standard elbow joint angle and standard shoulder joint angle are specific angle values. It is determined whether the extracted virtual projectile velocity value falls within the standard velocity range, and a velocity consistency flag is marked. If the virtual projectile velocity value falls within the interval, the velocity consistency flag is recorded as true; otherwise, it is recorded as false. The system determines whether the extracted virtual projectile deflection angle value exceeds the standard deflection tolerance range and marks it as an anomaly. If the virtual projectile deflection angle value exceeds the range, the anomaly is recorded as true; otherwise, it is recorded as false. Combining the weapon grip angle sequence, the system calculates the angle difference between the actual elbow joint angle, actual shoulder joint angle, and standard elbow joint angle in the current time frame. The absolute value of the angle difference is taken, and a posture deviation is marked. If the angle difference exceeds a preset 5-degree threshold, the posture deviation is recorded as true; otherwise, it is recorded as false. In some embodiments, the standard shooting posture sequence is established based on expert shooter data, and the standard value of each time frame in the sequence is accompanied by a statistically based allowable fluctuation range. It can be understood that the frame-by-frame comparison process is performed in real time in memory, and the generated label data is cached as intermediate results in a circular buffer for subsequent report generation.

[0045] The process of generating a shooting action analysis report containing action deviation markers and trajectory consistency ratings based on the comparison results is as follows: First, the proportion of time frames with a true velocity consistency marker is calculated to generate a trajectory velocity consistency rate, which is equal to the number of time frames with a true velocity consistency marker divided by the total number of time frames. Second, the proportion of time frames with a true deviation anomaly marker is calculated to generate a ballistic deviation frequency, which is equal to the number of time frames with a true deviation marker divided by the total number of time frames. Third, the proportion of time frames with a true attitude deviation marker is calculated to generate an attitude instability index, which is equal to the number of time frames with a true attitude deviation marker divided by the total number of time frames. The calculated trajectory velocity consistency rate, ballistic deviation frequency, and attitude instability index are input into a predefined rating mapping table. The rating mapping table outputs a trajectory consistency rating, which includes excellent, acceptable, and need-improvement levels. Finally, based on the changing patterns of the instrument grip angle sequence and trigger pressure sequence along the entire time axis, a pattern recognition algorithm identifies grip shake patterns, trigger pull patterns, or aiming lag patterns, and these identified patterns are added to the shooting action analysis report as action deviation markers. In some embodiments, the rating mapping table is implemented using a piecewise linear function or decision tree model. A trajectory velocity consistency rate higher than 90%, a ballistic deviation frequency lower than 5%, and an attitude instability index lower than 10% are mapped to an excellent level; a trajectory velocity consistency rate between 70% and 90%, a ballistic deviation frequency between 5% and 15%, and an attitude instability index between 10% and 25% are mapped to a satisfactory level; any indicator below the satisfactory threshold is mapped to a level requiring improvement. It can be understood that the identification of action deviation markers is based on the time-domain and frequency-domain feature analysis of the instrument grip angle sequence and trigger pressure sequence. The grip shaking mode is characterized by high-frequency, low-amplitude oscillations in the instrument grip angle sequence; the trigger snapping mode is characterized by a rapid rising edge in the trigger pressure sequence; and the aiming lag mode is characterized by changes in the instrument grip angle sequence continuously lagging behind changes in the standard angles in the standard shooting posture sequence.

[0046] In practice, the rating mapping table is calculated using a formula:

[0047] in: This represents the overall score used to determine trajectory consistency rating. Indicates the trajectory velocity consistency rate. Indicates the frequency of ballistic deviation. Indicates the instability index. These are pre-set positive weighting coefficients, each representing the importance of each indicator in the rating. Overall Score The calculated values ​​falling into different numerical ranges correspond to different trajectory consistency ratings. Optionally, data comparison is reflected in the conversion between the original label data and the derived evaluation indicators. The original binary label sequence is statistically aggregated into scalar percentages of trajectory velocity consistency rate, ballistic deviation frequency, and attitude instability index. These scalar indicators are further converted into discrete trajectory consistency ratings through formulas or mapping tables. The pattern recognition algorithm converts the continuous instrument grip angle sequence and trigger pressure sequence into one or more discrete action deviation labels. For example, an oscillation of the instrument grip angle sequence lasting 2 seconds with a frequency of 8 to 12 Hz is labeled as "moderate grip jitter". The generated shooting action analysis report is a structured document containing a textual description of the trajectory consistency rating and a list of action deviation labels.

[0048] See Figure 5 This is a bar chart comparing the performance of various components in a virtual training space. It evaluates the performance of the visual environment model, ballistic simulation engine, equipment motion library, and overall system using three core metrics: frame rate (FPS), latency (ms), and accuracy (%). The equipment motion library performs best in real-time performance, with the lowest latency and highest frame rate, which is crucial for the timeliness of motion capture and feedback in shooting training. Both the visual environment model and the equipment motion library achieve an accuracy of 99%, ensuring the accuracy of visual rendering and motion recognition. The overall system latency (18ms) is significantly higher than that of individual components, indicating that data interaction and synchronization between multiple components is the main performance bottleneck. The slight decrease in frame rate and accuracy also reflects the resource overhead and collaborative complexity during system integration.

[0049] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A shooting simulation training method based on virtual reality, characterized in that, include: Construct a virtual training space that includes a visual environment model, a ballistic simulation engine, and a library of weapon actions; Dynamic lighting and terrain perturbation variables are injected into the visual environment model; Activate the ballistic simulation engine and load the physical attribute templates corresponding to the virtual firing equipment; Start the equipment action library and load the standard shooting posture sequence; Perform environment initialization calibration to align the rendering output of the visual environment model, the initial state of the ballistic simulation engine, and the baseline data of the instrument action library. Capture the real-time gaze focus coordinates and body posture coordinates of the trainee in the visual environment model; Capture the sequence of grip angles and trigger pressure during the operation of a virtual shooting device; The real-time gaze focus coordinates, body posture coordinates, instrument grip angle sequence, and trigger pressure sequence are input into the ballistic simulation engine; The ballistic simulation engine generates a ballistic prediction dataset containing virtual projectile velocity curves and virtual projectile deflection angles based on physical property templates. The ballistic prediction dataset is compared frame by frame with the standard firing posture sequence in the weapon action library; Based on the comparison results, a shooting action analysis report is generated, which includes action deviation markers and trajectory consistency ratings.

2. The shooting simulation training method based on virtual reality according to claim 1, characterized in that, The steps of constructing a virtual training space that includes a visual environment model, a ballistic simulation engine, and a library of weapon actions specifically include: The three-dimensional scene mesh data and material mapping relationship of the virtual training space are generated using computer-aided design tools. Mark potential interaction areas and visually obstructed areas in the 3D scene mesh data; Bind the illumination intensity variation curve of the dynamic lighting variable and the surface height variation curve of the terrain disturbance variable to the corresponding vertices of the 3D scene mesh data; Configure virtual air density parameters, virtual gravity parameters, and virtual material penetration parameters for the ballistic simulation engine based on the physical property template; Based on the standard shooting posture sequence, posture constraint rules are constructed for the equipment action library, including elbow joint angle range, shoulder joint angle range, and spinal deviation tolerance.

3. The shooting simulation training method based on virtual reality according to claim 1, characterized in that, The execution environment initialization calibration steps specifically include: Send a calibration command to the visual environment model, triggering the visual environment model to output a calibration screen containing a specific color block array and scale indicators; Read the chromaticity values ​​of a specific color block array and the pixel dimensions of the scale marker in the calibration image; By comparing the chromaticity values ​​with the preset standard chromaticity threshold, color calibration parameters for the visual environment model are generated. By comparing pixel size with a preset standard scale threshold, size calibration parameters for the visual environment model are generated. Synchronize the color calibration parameters and size calibration parameters to the ballistic simulation engine and the instrument motion library; The ballistic simulation engine adjusts the virtual distance calculation scale in the physical property template based on the size calibration parameters; The instrument action library adjusts the posture judgment thresholds related to visual aiming in the standard shooting posture sequence based on color calibration parameters.

4. The shooting simulation training method based on virtual reality according to claim 1, characterized in that, The steps for capturing the trainee's real-time gaze focus coordinates and body posture coordinates in the visual environment model specifically include: Analyze the head orientation quaternion data and eye-tracking image data transmitted by the virtual reality headset; Convert the head direction quaternion data into an Euler angle dataset that includes pitch, yaw, and roll angles; Pupil contour recognition and corneal reflection point localization are performed on eye-tracking image data, and the coordinates of the real-time gaze focus in the three-dimensional coordinate system of the visual environment model are calculated by combining the Euler angle dataset. Analyzing the joint position data stream transmitted by the virtual reality body tracking kit; Based on the spatial positions of the shoulder, hip, and foot joints in the joint position data stream, body posture coordinates are obtained through inverse kinematics calculations. The body posture coordinates include the trunk tilt angle and the coordinates of the center of gravity projection point.

5. The shooting simulation training method based on virtual reality according to claim 1, characterized in that, The steps for capturing the sequence of instrument grip angles and trigger pressure during the operation of the virtual shooting instrument specifically include: Receives inertial measurement unit data and capacitive sensing array data transmitted from the virtual reality controller; Extract the real-time orientation matrix and angular velocity vector of the controller in the virtual training space from the inertial measurement unit data; The grip angle between the muzzle pointing axis and the standard pointing axis of the virtual shooting device is calculated based on the real-time orientation matrix and recorded in chronological order to form a sequence of grip angles. The capacitance changes of multiple sensing units covering the virtual trigger area are analyzed from the capacitive sensing array data. The changes in capacitance values ​​of multiple sensing units are weighted and fused to map them into continuous trigger pressure values, which are then recorded in chronological order to form a trigger pressure sequence.

6. The shooting simulation training method based on virtual reality according to claim 1, characterized in that, The steps of generating a ballistic prediction dataset containing virtual projectile velocity curves and virtual projectile deflection angles based on a physical property template by the ballistic simulation engine specifically include: The real-time gaze focus coordinates and body attitude coordinates are input into the initial conditions module of the ballistic simulation engine. The initial conditions module calculates the launch position vector and initial direction vector of the virtual projectile. The instrument grip angle sequence and trigger pressure sequence are input into the perturbation calculation module of the ballistic simulation engine. The perturbation calculation module, combined with the virtual air density parameter in the physical property template, calculates the initial directional perturbation vector caused by the unstable instrument grip and the initial velocity fluctuation caused by the unstable trigger pressure. The core solver of the ballistic simulation engine takes the launch position vector, initial direction vector, initial direction perturbation vector and initial velocity fluctuation as input, and combines the virtual gravity parameters and virtual material penetration parameters in the physical property template to calculate the trajectory of the virtual projectile in the virtual training space through numerical integration iteration. Extract the velocity magnitude and direction corresponding to each time step from the motion trajectory to form a virtual projectile velocity curve; The difference between the actual impact point coordinates of the virtual projectile and the theoretical aiming point coordinates is calculated to obtain the deflection angle of the virtual projectile.

7. The shooting simulation training method based on virtual reality according to claim 1, characterized in that, The step of comparing the ballistic prediction dataset with the standard firing posture sequence in the weapon action library frame by frame specifically includes: Synchronize and align the timeline of the ballistic prediction dataset with the timeline of the standard firing attitude sequence; For each synchronized time frame, extract the virtual projectile velocity value and virtual projectile deflection angle value corresponding to the time frame from the ballistic prediction dataset; Extract the standard velocity range, standard deflection tolerance interval, standard elbow joint angle, and standard shoulder joint angle corresponding to the time frame from the standard shooting posture sequence; Determine whether the virtual projectile's velocity value falls within the standard velocity range and mark it with a velocity consistency indicator; Determine whether the virtual projectile deflection angle exceeds the standard deflection tolerance range and mark the deviation as abnormal; By combining the sequence of grip angles of the device, calculate the angle difference between the actual elbow joint angle and the actual shoulder joint angle of the current frame and the standard value, and mark the posture deviation.

8. The shooting simulation training method based on virtual reality according to claim 7, characterized in that, The step of generating a shooting action analysis report containing action deviation markers and trajectory consistency ratings based on the comparison results specifically includes: Calculate the proportion of time frames in which the velocity consistency flag is true, and generate the trajectory velocity consistency rate. Calculate the proportion of deviation anomalies marked as true across all time frames to generate ballistic deviation frequency; The proportion of attitude deviation indicators exceeding a preset threshold in all time frames is counted to generate an attitude instability index. The trajectory velocity consistency rate, ballistic deviation frequency, and attitude instability index are input into a predefined rating mapping table. The rating mapping table outputs a trajectory consistency rating, which includes excellent, qualified, and need-to-improve levels. Based on the changing patterns of the instrument grip angle sequence and trigger pressure sequence, the grip shake pattern, trigger snap pattern, or aiming lag pattern are identified, and the identified patterns are added to the shooting action analysis report as action deviation markers.

9. The shooting simulation training method based on virtual reality according to claim 1, characterized in that, The method also includes a dynamic environment response step in the virtual training space: The visual environment model calculates collision events between virtual projectiles and scene objects in real time based on the trajectory of virtual projectiles in the ballistic prediction dataset. When a collision event is triggered, the visual environment model generates a bullet hole indentation geometric deformation and a corresponding fragment particle effect on the 3D scene mesh data based on the virtual material penetration parameters in the physical property template. Meanwhile, the visual environment model sends the location coordinates and collision intensity values ​​of the collision event to the ballistic simulation engine; The ballistic simulation engine updates the physical property template of subsequent virtual projectiles based on the collision intensity value. The update includes adjusting the virtual air density parameter to simulate the effect of dust, or adjusting the virtual gravity parameter to simulate shock wave disturbance.

10. A shooting simulation training system based on virtual reality, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the shooting simulation training method based on virtual reality as described in any one of claims 1 to 9.

Images