Visual and somatosensory synchronous control method and device for VR driving simulation

Abstract

The application discloses a visual and somatosensory synchronous control method and device for VR driving simulation, and belongs to the technical field of VR virtual driving. The method comprises the following steps: periodically measuring the time delay parameters of a visual rendering link and a platform mechanical execution link; taking the two time delay parameters as the prediction time domains of the two links respectively, and establishing a unified target time; based on driving operation input and a vehicle dynamics model, outputting visual prediction states and platform prediction states corresponding to the prediction time domains; based on the visual prediction states and the platform prediction states, generating visual rendering target data and platform control target data respectively; triggering the execution of the two target data at the same time; collecting platform IMU linear acceleration and angular velocity, comparing the theoretical somatosensory target states corresponding to the platform control target data to obtain synchronization errors, generating outer loop feedback correction amounts through proportion-integration, superimposing the feedback correction amounts to feedforward control amounts to form comprehensive control instructions, and driving the platform to output somatosensory movements synchronized with visual pictures.

Description

Technical Field

[0001] This invention belongs to the fields of virtual reality simulation, motion simulation control, electromechanical servo control and real-time synchronous control technology, and specifically relates to a visual and haptic synchronous control method and device for VR driving simulation. Background Technology

[0002] Existing VR virtual driving systems typically include a driver input acquisition device, a vehicle dynamics model, a graphics rendering unit, a head-mounted display device, and a six-degree-of-freedom mechanical motion platform. The driver's steering wheel, accelerator, brake, and gear shifting operations are collected and input into the vehicle dynamics model. The vehicle state output from the model is used to drive VR scene rendering and, through a rendering algorithm, converted into motion control commands for the motion platform. This allows the driver to experience corresponding posture changes and acceleration sensations while viewing the virtual scene.

[0003] However, in practical engineering applications, the visual rendering link and the motion platform servo link are not considered dynamically equivalent links. The visual link is usually affected by factors such as graphics rendering thread scheduling, GPU graphics pipeline caching, display refresh cycle, and head-mounted display posture fusion latency; the platform link is usually affected by factors such as control cycle, industrial bus communication latency, servo driver response, actuator inertia, platform structural compliance, and travel, speed, and acceleration limitations.

[0004] Therefore, even if the visual and platform links originate from the same vehicle status data, problems often arise such as the visual image changing before the platform's haptic feedback lags behind, the platform starting to return to center while the visual image is still in a phase of intense motion, and inconsistencies in the spectral distribution between vision and haptic feedback. These issues lead to deviations in the driver's perception of the direction, intensity, and rhythm of movement, and in severe cases, can cause dizziness, enhanced illusions, and training distortion. To address these problems, the shortcomings of existing technologies are mainly reflected in the following four aspects: 1. Simply achieving "simultaneous data transmission" does not achieve "synchronous perception." Most existing solutions only output rendering data and platform control data at the same time in the application layer, or calculate the two data streams using the same cycle. However, due to the different latency and dynamic response characteristics of the visual link and the platform link, "simultaneous transmission" does not equate to the driver "simultaneous perception."

[0005] 2. Traditional motion rendering algorithms primarily focus on motion reproduction without controlling for synchronization errors between visual and haptic feedback. Existing motion rendering algorithms mainly focus on approximating the feeling of vehicle motion within a limited mechanical space through high-pass and low-pass filtering, tilt coordination, and centering strategies. Their optimization goals are primarily platform executability and motion realism, rather than controlling the synchronization error between visual and haptic output.

[0006] 3. Platform feedback typically focuses on the actuator side, failing to accurately reflect the actual occupant experience. While some existing platform control systems incorporate servo motor encoder feedback, this type of feedback primarily reflects the theoretical displacement or joint state on the motor side, making it difficult to accurately characterize the actual inertial state output by the platform's top plate. Consequently, it cannot fully reflect the impact of mechanical clearances, compliance, load variations, and structural vibrations on the occupant's actual perception.

[0007] 4. Fixed parameter control struggles to accommodate different driving scenarios. For example, off-road bumpy scenarios require the retention of more high-frequency impact characteristics, while high-speed cruising scenarios emphasize smoothness and stability. Emergency braking and continuous curves also have different requirements for attitude changes and acceleration / deceleration matching. Fixed parameter shuffling algorithms struggle to balance synchronization, realism, and safety across multiple scenarios. Summary of the Invention

[0008] The core problem that this invention aims to solve is not simply to improve the platform's tracking accuracy, nor simply to optimize the washing algorithm, but rather to address complex technical issues such as the inherent time delays of the visual link and the platform link, the limitations of the platform's mechanical travel and safety thresholds, the difficulty in simultaneously taking into account both time delay compensation and execution error correction when relying solely on prediction or solely on feedback, and the different effective somatosensory spectrum distributions in different scenarios.

[0009] This invention proposes a method and device for synchronized visual and haptic control in VR driving simulation, which enables visual output and haptic output of the mechanical motion platform to be generated collaboratively for the same future target. Combined with the actual inertial feedback of the platform and scene adaptive parameter adjustment, it achieves highly consistent visual and haptic synchronization within the platform's limited space.

[0010] In a first aspect, the present invention proposes a method for simultaneous visual and haptic control in VR driving simulation, comprising: periodically measuring the first time delay parameter of the visual rendering link. The second delay parameter of the platform mechanical execution link ; with the first time delay parameter and the second delay parameter These are respectively used as the prediction time domains for the visual rendering link and the platform mechanical execution link, and are based on the current control time. Add the first delay parameter and the second delay parameter The maximum value in the time frame is used as the unified target time. Based on the current driving operation input and the vehicle dynamics model, the visual prediction state corresponding to the prediction time domain is output respectively. and platform prediction status Based on the visual prediction state Generate visual rendering target data and predict the state based on the platform. Generate platform control target data, which serves as the theoretical perceived target state, including the platform's desired linear acceleration and desired angular velocity. The desired linear acceleration and desired angular velocity are determined by the platform's predicted state. The platform feedforward control quantity is obtained after washing. Then, through platform kinematic mapping, both the visual rendering target data and the platform control target data are obtained with the unified target time. As a time coordinate alignment reference; simultaneously triggering the execution of the visual rendering target data and the platform control target data with a unified time base signal; real-time acquisition of the linear acceleration and angular velocity output by the platform's IMU, comparing with the theoretical tactile target state to obtain the synchronization error vector. The synchronization error vector The outer loop feedback correction quantity is generated through proportional-integral conversion. The outer loop feedback correction amount Superimposed on the platform feedforward control quantity To form integrated control commands This drives the platform's mechanical execution link output to synchronize with the visual image, enabling haptic motion.

[0011] In some examples, the first delay parameter Defined as the time interval from the issuance of the control command to the effective display output; the second delay parameter Defined as the time interval from the issuance of a control command to the first time the linear acceleration magnitude or angular velocity magnitude output by the IMU exceeds a preset threshold; the first time delay parameter measured within multiple consecutive control cycles. and the second delay parameter Perform a moving average or weighted update.

[0012] In some examples, the unified target time It serves only as a time alignment anchor point and is not directly used as the prediction time-domain input for the visual rendering link and the platform mechanical execution link.

[0013] In some examples, the visual rendering target data and the platform control target data are both derived from the same vehicle dynamics model, the same current control input, and the first time delay parameter updated within the same control cycle. and the second delay parameter Driver generation.

[0014] In some examples, the integrated control commands satisfy:

[0015] in, These are the feedforward weighting coefficients; This represents the outer loop feedback weighting coefficient.

[0016] In some examples, the current driving condition category is identified, and adjustments are made dynamically based on the driving condition category. , And the parameters of the washing algorithm.

[0017] In some examples, when the platform approaches its physical limits, the theoretical haptic target state is decomposed into low-frequency components. and high frequency components , in mapping ratio r Control the low-frequency somatosensory component The mapping ratio is obtained by replacing direct translation with a tilted posture. r The proportion of constraints as the characterization platform approaches its physical limits α The low-frequency component changes continuously. The remaining part and the high-frequency component Execute within the remaining range.

[0018] In some examples, the constraint percentage α The mapping ratio is calculated based on the ratio of the actual amount of motion in the current restricted degree of freedom to the upper limit of the safe working range of that degree of freedom. r ∈[0,1], when the constraint accounts for α Below the startup threshold α When 0, the mapping ratio r =0; when the constraint percentage α Located in the interval [ α When [0, 1], the mapping ratio r With the proportion of the constraints α monotonically increasing to r max , r max The value ranges from 0.7 to 0.9.

[0019] Secondly, this invention proposes a vision and haptic synchronization control device for VR driving simulation, comprising: a heterogeneous link delay online calibration module, which is configured to periodically measure the first delay parameter of the visual rendering link. The second delay parameter of the platform mechanical execution link The prediction time domain determination and unified target time establishment module is configured to use the first time delay parameter. and the second delay parameter These are respectively used as the prediction time domains for the visual rendering link and the platform mechanical execution link, and are based on the current control time. Add the first delay parameter and the second delay parameter The maximum value in the time frame is used as the unified target time. The future vehicle motion state prediction module is configured to output the visual prediction state corresponding to the prediction time domain based on the current driving operation input and the vehicle dynamics model. and platform prediction status The visual rendering target generation module is configured to use the same vehicle dynamics model and the same current control input, with the unified target time. Based on the aforementioned visual prediction state Generate visual rendering target data; the platform controls the target generation module, which is configured to be based on the same vehicle dynamics model and the same current control input, with the unified target time. Based on the platform's predicted state, the baseline is used. Generate platform control target data, wherein the platform control target data serves as the theoretical somatosensory target state, including the platform's desired linear acceleration and desired angular velocity, which are determined by the platform's predicted state. The platform feedforward control quantity is obtained after washing. The synchronization error vector is then obtained through platform kinematic mapping; the platform execution end synchronization error compensation module is configured to collect the linear acceleration and angular velocity output by the platform's IMU in real time, and compare them with the theoretical somatosensory target state to obtain the synchronization error vector. The synchronization error vector The outer loop feedback correction quantity is generated through proportional-integral conversion. The outer loop feedback correction amount Superimposed on the platform feedforward control quantity To form integrated control commands This is to drive the platform's mechanical execution link output to synchronize with the visual image for haptic motion; the frequency domain redistribution processing module is configured to decompose the theoretical haptic target state into low-frequency components when the platform approaches its physical limits. and high frequency components , in mapping ratio r Control the low-frequency somatosensory component The mapping ratio is obtained by replacing direct translation with a tilted posture. r The proportion of constraints as the characterization platform approaches its physical limits α The low-frequency component changes continuously. The remaining part and the high-frequency component Execute within the remaining range.

[0020] This invention significantly reduces the equivalent perceptual time lag between vision and somatosensory perception. By using a mechanism for generating future states from a common source and for synchronously disseminating information based on a unified time base, the visual link and the platform link can work together to output information around the same future target moment, thereby reducing the driver's visual-vestibular asynchrony.

[0021] This invention improves the consistency between the platform's actual output and the theoretical target. By introducing actual inertial feedback from the moving platform to construct closed-loop control, output deviations caused by mechanical backlash, load variations, servo lag, and structural compliance can be effectively reduced.

[0022] This invention improves the synchronization quality and haptic realism in different driving scenarios. Through scene recognition and parameter adaptation mechanisms, the same platform can achieve more scene-specific haptic output in scenarios such as off-road bumps, continuous curves, high-speed cruising, and emergency braking.

[0023] This invention improves the executability and continuity of motion within a limited workspace. Through a redistribution mechanism under limit constraints, it reduces the abrupt changes, distortions, and discomfort caused by traditional hard limiting.

[0024] This invention balances synchronization, realism, and security. Through feedforward and feedback coordination and online parameter adjustment, a balance is achieved between visual and haptic synchronization, haptic realism, and platform security constraints. Attached Figure Description

[0025] Figure 1 This is a flowchart of a VR driving simulation vision and motion-sensory synchronization control method according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of online calibration of heterogeneous delay between visual and mechanical links according to an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram illustrating the prediction time domain determination and unified target time establishment for each link according to an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram of a dual-loop control structure for the platform execution end according to an embodiment of the present invention.

[0029] Figure 5 This is a schematic diagram of frequency domain redistribution and mapping scaling function under platform physical constraints according to an embodiment of the present invention.

[0030] Figure 6 This is a block diagram of a VR driving simulation vision and haptic synchronization control device according to an embodiment of the present invention. Detailed Implementation

[0031] The VR virtual driving system (hereinafter referred to as the "system") includes a six-degree-of-freedom motion platform, a driving simulation cabin, a driving operation input module, a VR display device, a platform posture sensing module, a platform servo controller, and a main control computer.

[0032] The six-DOF motion platform adopts a Stewart parallel platform, which includes a fixed base, a moving platform (hereinafter referred to as the platform), and six electric servo actuators (also known as actuators). Each actuator is connected to the fixed base and the moving platform through ball joints, Hooke joints, or other universal joints to output a composite motion with three translational and three rotational degrees of freedom.

[0033] The driving simulator is mounted on a moving platform to carry the driver and is equipped with a steering wheel, pedals, gear shift mechanism, safety restraint devices and other interactive equipment.

[0034] The driving operation input module is used to collect steering wheel angle, throttle opening, brake input, gear status and other driving control signals.

[0035] VR display devices are used to show drivers a scene corresponding to the current virtual driving state. VR display devices can be head-mounted displays, panoramic displays, or other immersive visual display devices.

[0036] The platform pose sensing module is located on the moving platform body or at a position rigidly connected to the moving platform, such as on the top plate of the moving platform, and is used to acquire the actual motion state of the moving platform. The platform pose sensing module includes at least an inertial measurement unit (IMU) for outputting the platform's actual linear acceleration, angular velocity, attitude angle, linear velocity, and / or position state.

[0037] The platform servo controller (also known as a servo driver) is connected to six servo actuators to drive the movement of each actuator according to platform control commands.

[0038] The main control computer communicates with the driving operation input module, VR display device, platform pose sensing module and platform servo controller to complete vehicle state prediction, visual target generation, platform control quantity calculation, synchronization error construction, scene recognition, parameter adaptation and unified time base synchronization.

[0039] This invention establishes a unified time base for predicting future vehicle states, a closed-loop platform inertial feedback mechanism, and a scene-adaptive parameter adjustment mechanism. This enables VR visual targets and platform haptic targets to no longer be generated independently, but to be collaboratively output towards the same future moment. This reduces the equivalent perception time difference between vision and hapticity, and improves the synchronicity, immersion, realism, and comfort during virtual driving.

[0040] The visual target and the platform target both originate from the same predicted vehicle state at a future moment. Instead of generating separate visual and platform outputs based on the current vehicle state, the main control computer predicts the vehicle state at the future control moment based on the driver input and the vehicle dynamics model, and uses this future state as the common driving source for both the visual rendering target and the platform feedforward control target.

[0041] The feedback object is the actual inertial state of the moving platform, not just the theoretical state of the actuator. The synchronization error is not based on the theoretical displacement on the motor encoder side, but on the actual linear acceleration, angular velocity, attitude angle, velocity and displacement measured on the moving platform, so that the feedback quantity is closer to the actual perception of the occupants.

[0042] This invention simultaneously introduces predictive feedforward, error feedback, and scene adaptive parameter adjustment within the same control closed loop. In other words, it does not simply stack predictive control, feedback control, and scene recognition in parallel, but rather couples these three elements together in a unified solution within the same control loop, all centered around minimizing the synchronization error between vision and haptic feedback.

[0043] Constraint redistribution is performed when the platform approaches its physical limits. When a degree of freedom approaches its travel, velocity, acceleration, or attitude limits, instead of a hard cutoff, some non-executable translational acceleration components are mapped to the rotational degree of freedom through tilt coordination or other means to maintain the continuity of occupant perception.

[0044] Vision and platform commands are issued synchronously at a unified time-base trigger point. Through a unified time-base scheduling mechanism, vision targets and platform final control commands are output at the same control trigger point, reducing jitter and misalignment caused by differences in multi-threaded scheduling, device buffering, and communication latency.

[0045] Figure 1 A flowchart illustrating a method for synchronized visual and haptic control in VR driving simulation is provided. This method includes: reading the current driving operation input and vehicle status; and performing online calibration of heterogeneous link latency, which includes periodically measuring the first latency parameter of the visual rendering link. The second delay parameter of the platform mechanical execution link ; with the first time delay parameter and the second delay parameter These are respectively used as the prediction time domains for the visual rendering link and the platform mechanical execution link, and are based on the current control time. Plus The corresponding future moment is used as the unified target moment. Based on the current driving operation input and the vehicle dynamics model, the visual prediction state corresponding to the prediction time domain is output respectively. and platform prediction status Based on the visual prediction state Generate visual rendering target data and predict the state based on the platform. Generate platform control target data, which serves as the theoretical perceived target state, including the platform's desired linear acceleration and desired angular velocity. The desired linear acceleration and desired angular velocity are determined by the platform's predicted state. The platform feedforward control quantity is obtained after washing. Then, through platform kinematic mapping, both the visual rendering target data and the platform control target data are obtained with the unified target time. As a time coordinate alignment reference; simultaneously triggering the execution of the visual rendering target data and the platform control target data with a unified time base signal; real-time acquisition of the linear acceleration and angular velocity output by the platform's IMU, comparing with the theoretical tactile target state to obtain the synchronization error vector. The synchronization error vector The outer loop feedback correction quantity is generated through proportional-integral conversion. The outer loop feedback correction amount Superimposed on the platform feedforward control quantity To form integrated control commands This method drives the platform's mechanical execution link output to synchronize with the visual image, enabling haptic motion. The method is described in detail below.

[0046] Step 1: Online calibration of heterogeneous link latency.

[0047] like Figure 2 As shown, the system periodically sends out test trigger signals to measure the first latency parameter of the visual rendering link. The second delay parameter of the motion platform mechanical execution link First delay parameter Second delay parameter The calibration goal is to make the arrival time of visual output and tactile output consistent at the driver's perception level, rather than just aligning them at the moment the command is issued.

[0048] Specifically, the first time delay parameter Defined as the time interval from the issuance of the control command to the effective display output (effective driver visual perception), the response time is the moment when the target test frame completes the display output and the driver can perceive the visual change.

[0049] Second delay parameter Defined as the time interval from the issuance of a control command to the detection of a motion response by the IMU on the platform's top panel, the response time is defined as the moment when the linear acceleration or angular velocity magnitude output by the IMU on the platform's top panel first exceeds a preset threshold and the driver can perceive a change in motion. The preset threshold can be determined based on the stationary noise baseline and the lower limit of the motion amplitude to balance response sensitivity and noise immunity.

[0050] To suppress fluctuations in a single measurement, measurements taken over multiple consecutive control cycles are processed. and Alternatively, a sliding average or weighted update can be performed. The typical sliding window length is 10 to 50 control periods, or an exponential weighted update method that assigns higher weight to recent measurements can also be used.

[0051] Step 2: Determine the time domain for each link prediction and establish a unified target time.

[0052] like Figure 3 As shown, the measurements obtained in step 1 and These are used as the prediction temporal domains for the visual link and the mechanical link, respectively. The visual link uses... As a lead time for rendering target prediction, mechanical links are used... As a lead time, it is used to predict platform control targets.

[0053] At the same time, the current control time Plus (i.e., the first time delay parameter) Second delay parameter The future moment corresponding to the maximum value in the [value] is used as the unified target moment for visual output and tactile output. This serves as the common time coordinate anchor point for both targets. Follow , The dynamic updates and control cycles are updated periodically.

[0054] Under this implementation approach, each link uses its own measured latency as the compensation benchmark to avoid leading deviations in links with shorter latency when using a uniform maximum compensation amount; and a uniform target time is used. It is used only as a time alignment anchor point to determine whether two targets are facing the same reference time, and is not directly used as the prediction time domain input for any link.

[0055] Step 3: Predict future vehicle motion state.

[0056] Based on the current driving operation input and vehicle dynamics model, respectively using and As the prediction time domain, the vehicle motion state at the corresponding moment is predicted. The driving operation input includes at least the steering wheel angle, throttle opening, and braking signal; the predicted vehicle motion state includes at least one or more of the following: attitude, velocity, linear acceleration, and angular velocity.

[0057] The state predictor employs an Extended Kalman Filter (EKF), but can also use Model Predictive Control (MPC) or other short-time prediction methods based on vehicle dynamics models. The predictor reads the updated state data in each control cycle. , Output the visual prediction state respectively and platform prediction status .

[0058] Step 4: Visual rendering target and platform control target are generated from the same source.

[0059] Based on the visual prediction state obtained in step 3 and platform prediction status Simultaneously, visual rendering target data and platform control target data are generated; both targets are set at a unified target time. As a time coordinate alignment benchmark, it ensures that the visual output and the moment of tactile sensation are consistent.

[0060] The "co-source generation" mentioned in this invention refers to the fact that both the visual rendering target data and the platform control target data are generated by the same vehicle dynamics model, the same current control input, and the link delay parameters updated within the same control cycle. In the dual prediction time domain implementation, the two correspond to the future states under different prediction time domains, but share a unified time alignment reference and the same control source data, thereby reducing the accumulation of phase deviation from the source.

[0061] Visual rendering target data is used to drive the visual link to perform advance rendering. The platform controls the target data based on the platform's prediction state. The platform feedforward control quantity is obtained after washing. Then, the desired linear acceleration and desired angular velocity of the platform top plate are obtained through platform kinematic mapping, which are used as the theoretical somatic target state.

[0062] Step 5: Unified time base synchronization trigger.

[0063] The execution of visual rendering target data and platform control target data is triggered simultaneously using a unified time base signal. Preferably, both the visual frame buffer exchange signal and the platform control interrupt signal are driven by the same global time base heartbeat to reduce discrete errors introduced by multi-threaded scheduling, buffer queuing, and inconsistencies in subsystem scheduling timing. The unified time base can be provided by a master computer hardware timer interrupt, a real-time bus synchronization clock (e.g., an EtherCAT distributed clock), or an independent synchronization clock module.

[0064] Step 6: Platform execution end synchronization error double-loop closed-loop correction.

[0065] The control architecture of this invention consists of a feedforward channel and dual feedback loops (inner encoder loop and outer IMU synchronization error compensation loop). The feedforward channel is based on the expected motion calculated by the predictive open-loop platform in step 3, the inner loop ensures the basic tracking of the actuator, and the outer loop compensates for the haptic deviation at the actuator end. The feedforward and feedback control quantities are superimposed to form the final control command.

[0066] The platform employs a dual-loop control structure: the inner loop uses feedback from the actuator encoder to form a position or velocity loop, ensuring the actuator's basic tracking capability; the outer loop uses an IMU mounted on the platform's top plate to form a synchronization error compensation loop. This loop can directly measure the actual inertial state of the moving platform, effectively compensating for mechanical backlash, structural compliance, and perceptual deviations caused by load disturbances that are difficult for the actuator encoder to reflect. Because the IMU is directly installed on the moving platform perceived by the occupants, its measurements of linear acceleration and angular velocity are closer to the driver's actual perceptual experience, rather than relying solely on the theoretical state of the motor-side encoder in traditional solutions.

[0067] like Figure 4 As shown, the outer loop operates as follows: It collects the linear acceleration and angular velocity output from the IMU on the platform's top plate in real time, compares this with the theoretical haptic target state generated in step 4, and obtains the synchronization error vector. Then process it using a proportional-integral control law. Generate outer loop feedback correction quantity ;Will Superimposed on the platform feedforward control quantity after washing process To form integrated control commands And send it to the servo driver.

[0068] In a preferred embodiment, the platform's final control command The following relationship must be satisfied:

[0069] in, These are the feedforward weighting coefficients; For the outer loop feedback weighting coefficient, The value is dynamically adjusted based on the magnitude of the synchronization error and the system calibration results, with a typical range of 0.3 to 0.8. When the synchronization tracking error is small and the platform is running stably, the value can be appropriately reduced. ,improve At this point, feedforward control is mainly relied upon to ensure response speed; when the synchronization tracking error continues to increase or the actual output of the platform deviates significantly, the speed will be increased accordingly. This is to enhance the closed-loop correction capability.

[0070] Step 7: Scene recognition and adaptive adjustment of control parameters.

[0071] This step is an optional optimized implementation method, which identifies the current driving condition and adjusts the feedforward weight coefficients according to the condition category. Feedback weight coefficient The goal is to achieve a reasonable balance between synchronization, haptic realism, and platform workspace utilization by refining the algorithm parameters.

[0072] Supported operating conditions include off-road bumps, high-speed cruising, continuous steering, and emergency braking. This step is a supplementary optimization to the main synchronization control process.

[0073] Step 8: Frequency domain reallocation under platform physical constraints.

[0074] When the platform approaches its physical limits, a frequency domain redistribution mechanism with continuously varying mapping ratios replaces hard truncation, allowing low-frequency and high-frequency components to be rationally distributed within the available motion range, reducing abrupt changes in somatosensory perception and maintaining a continuous correspondence between vision and somatosensory perception.

[0075] To describe how close the platform is to its physical limits, a constraint percentage is defined. α Constraint percentage α The calculation is based on the ratio of the actual motion of the current restricted degree of freedom to the upper limit of the safe working range of that degree of freedom. In a preferred embodiment, the ratio of the current absolute value of the platform's translational displacement, attitude angle, and angular acceleration to the corresponding safety threshold is calculated, and the maximum value of the three is taken as the current value. α .

[0076] To control the tilt coordination intensity of low-frequency components, a mapping ratio is defined. r Mapping ratio r Indicates low-frequency somatosensory components The proportion of tilting postures instead of direct translations is used in the middle. r ∈[0,1]; r =0 indicates no tilt substitution, and r=1 indicates that all low-frequency components are output in a tilted manner to achieve the equivalent body sensation.

[0077] When the motion in any degree of freedom of the platform reaches a preset threshold within a safe range, frequency domain reallocation is initiated. First, the predicted somatosensory signal (i.e., the theoretical somatosensory target state) is decomposed into low-frequency components. and high frequency components The cutoff frequency can be set from 1Hz to 3Hz depending on the platform's responsiveness.

[0078] Mapping ratio r With the proportion of constraints α Continuous change: when α Below the startup threshold α At 0 o'clock,r =0; when α Located in the interval [ α When [0, 1], r Follow α monotonically increasing to r max The monotonically increasing function can be a piecewise linear function or a sigmoid function. r max Typical values ​​range from 0.7 to 0.9.

[0079] Based on this, the proportion of low-frequency components is r The portion is output through the platform's tilt posture, using the gravity component to create an equivalent tactile sensation; the remaining (1- r The low-frequency and high-frequency components of the part Prioritize the execution within the remaining range of motion to preserve the impact and jolt characteristics as much as possible.

[0080] The high and low frequency components are connected by a smooth transition mechanism. The transition time constant is preferably 50ms to 200ms to avoid hard cutoff under extreme conditions and maintain the continuity of the haptic output and its correspondence with the visual image.

[0081] When the system identifies high-frequency components in the vehicle's longitudinal and vertical accelerations, and the rate of change of suspension displacement increases, it determines that the current scenario is an off-road bumpy ride. In this case, the system increases the high-frequency channel retention capability and feedforward gain to enhance the reproduction capability of short-term impacts and vertical bumps, while limiting the accumulation of continuous large displacements and excessive attitude changes to prevent the platform from reaching its mechanical limits.

[0082] When the vehicle speed is maintained at a high level and the steering input is relatively smooth, it is determined to be a high-speed cruising scenario. At this time, the strength of the smoothness-related low-pass filter is increased to reduce the output of ineffective high-frequency vibrations, so that both the visual perception and the platform feel remain stable, thereby improving the comfort of long-distance driving.

[0083] When a rapid increase in braking input is detected and a significant increase in predicted vehicle longitudinal deceleration is observed, an emergency braking scenario is identified. The system enhances the longitudinal deceleration feedforward and pitch attitude coordination output, enabling the driver to achieve greater consistency between visual deceleration changes and the perceived forward lean of the body.

[0084] Figure 6 A block diagram of a VR driving simulation's visual and haptic synchronized control device is shown. (Example) Figure 6 As shown, the device includes: a heterogeneous link delay online calibration module, a prediction time domain determination and unified target time establishment module, a future vehicle motion state prediction module, a visual rendering target generation module, a platform control target generation module, a scene recognition and parameter adaptation module, a platform execution end synchronization error compensation module, and a frequency domain redistribution processing module.

[0085] The heterogeneous link latency online calibration module is configured to perform the heterogeneous link latency online calibration described in step 1, periodically issuing test trigger signals to measure the first latency parameter of the visual rendering link. The second delay parameter of the motion platform mechanical execution link The measurement results are then updated using a moving average or weighted average.

[0086] The prediction time domain determination and unified target time establishment module is configured to perform the prediction time domain determination and unified target time establishment for each link as described in step 2, in order to and These are used as the prediction time domains for the visual link and the mechanical link, respectively, and a unified target time is established. As a common time coordinate anchor point.

[0087] The future vehicle motion state prediction module is configured to perform the future vehicle motion state prediction described in step 3, employing an extended Kalman filter, based on the current driving operation input and the vehicle dynamics model, and respectively using... and As the prediction time domain, it predicts the vehicle motion state at the corresponding time and outputs the visual prediction state. and platform prediction status .

[0088] The visual rendering target generation module and the platform control target generation module are respectively configured to perform the co-source generation of visual rendering targets and platform control targets as described in step 4, based on the same vehicle dynamics model, the same current control input, and the link delay parameters updated within the same control cycle, so as to unify the target time. Based on this, visual rendering target data and platform control target data are generated respectively.

[0089] The platform execution end synchronization error compensation module is configured to perform the platform execution end synchronization error dual-loop closed-loop correction described in step 6, including an inner loop position / velocity control loop and an outer loop synchronization error compensation loop. The outer loop performs error compensation on the theoretical tactile target state based on the actual inertial state fed back by the platform top plate IMU, generating an outer loop feedback correction amount. and feedforward control quantity The final control command is formed by superimposing these commands.

[0090] The frequency domain redistribution processing module is configured to perform frequency domain redistribution under the physical constraints of the platform as described in step 8. When the platform approaches its physical limits, the predicted somatosensory signal is decomposed into low-frequency and high-frequency components, and each component is reasonably allocated within the available motion range through a mechanism of continuous change in mapping ratio.

[0091] The scene recognition and parameter adaptation module is configured to perform the scene recognition and control parameter adaptive adjustment described in step 7, identify the current driving condition category, and dynamically adjust the feedforward control weight, feedback control weight, and washout algorithm parameters.

[0092] All modules work synchronously under the coordination of a unified time base signal, ensuring that the arrival time of visual output and tactile output at the driver's perception level tends to be consistent.

Claims

1. A method for simultaneous visual and haptic control in VR driving simulation, characterized in that, include: Periodically measure the first latency parameter of the visual rendering link The second delay parameter of the platform mechanical execution link ; Using the first time delay parameter and the second delay parameter These are respectively used as the prediction time domains for the visual rendering link and the platform mechanical execution link, and are based on the current control time. Add the first delay parameter and the second delay parameter The maximum value in the time frame is used as the unified target time. ; Based on the current driving operation input and the vehicle dynamics model, the visual prediction state corresponding to the prediction time domain is output respectively. and platform prediction status ; Based on the visual prediction state Generate visual rendering target data and predict the state based on the platform. Generate platform control target data, which serves as the theoretical perceived target state, including the platform's desired linear acceleration and desired angular velocity. The desired linear acceleration and desired angular velocity are determined by the platform's predicted state. The platform feedforward control quantity is obtained after processing by the washing algorithm. Then, through platform kinematic mapping, both the visual rendering target data and the platform control target data are obtained with the unified target time. Used as a time coordinate alignment reference; The execution of the visual rendering target data and the platform control target data is triggered simultaneously using a unified time base signal; The linear acceleration and angular velocity output by the IMU of the real-time acquisition platform are compared with the theoretical tactile target state to obtain the synchronization error vector. The synchronization error vector The outer loop feedback correction quantity is generated through proportional-integral conversion. The outer loop feedback correction amount Superimposed on the platform feedforward control quantity To form integrated control commands This drives the platform's mechanical execution link output to synchronize with the visual image, enabling haptic motion.

2. The VR driving simulation visual and haptic synchronization control method according to claim 1, characterized in that, First delay parameter Defined as the time interval from the issuance of the control command to the effective display output; the second delay parameter Defined as the time interval from the issuance of a control command to the first time the linear acceleration magnitude or angular velocity magnitude output by the IMU exceeds a preset threshold; the first time delay parameter measured within multiple consecutive control cycles. and the second delay parameter Perform a moving average or weighted update.

3. The VR driving simulation visual and haptic synchronization control method according to claim 1, characterized in that, The unified target time It serves only as a time alignment anchor point and is not directly used as the prediction time-domain input for the visual rendering link and the platform mechanical execution link.

4. The VR driving simulation visual and haptic synchronization control method according to claim 1, characterized in that, Both the visual rendering target data and the platform control target data are derived from the same vehicle dynamics model, the same current control input, and the first time delay parameter updated within the same control cycle. and the second delay parameter Driver generation.

5. The VR driving simulation visual and haptic synchronization control method according to claim 1, characterized in that, The integrated control command satisfy: in, These are the feedforward weighting coefficients; This represents the outer loop feedback weighting coefficient.

6. The VR driving simulation visual and haptic synchronization control method according to claim 5, characterized in that, Identify the current driving condition category and dynamically adjust the feedforward weight coefficients based on the category. Outer loop feedback weighting coefficient And the parameters of the shuffling algorithm.

7. The VR driving simulation visual and haptic synchronization control method according to claim 1, characterized in that, When the platform approaches its physical limits, the theoretical tactile target state is decomposed into low-frequency components. and high frequency components , in mapping ratio r Control the low-frequency component The mapping ratio is obtained by replacing direct translation with a tilted posture. r The proportion of constraints as the characterization platform approaches its physical limits α The low-frequency component changes continuously. The remaining part and the high-frequency component Execute within the remaining range.

8. The VR driving simulation visual and haptic synchronization control method according to claim 7, characterized in that, The proportion of constraints α The mapping ratio is calculated based on the ratio of the actual amount of motion in the current restricted degree of freedom to the upper limit of the safe working range of that degree of freedom. r ∈[0,1], when the constraint accounts for α Below the startup threshold α When 0, the mapping ratio r =0; when the constraint percentage α Located in the interval [ α When [0, 1], the mapping ratio r With the proportion of the constraints α monotonically increasing to r max .

9. A VR driving simulation vision and motion-sensory synchronous control device, characterized in that, include: The heterogeneous link latency online calibration module is configured to periodically measure the first latency parameter of the visual rendering link. The second delay parameter of the platform mechanical execution link ; The prediction time-domain determination and unified target time establishment module is configured to use the first time delay parameter. and the second delay parameter These are respectively used as the prediction time domains for the visual rendering link and the platform mechanical execution link, and are based on the current control time. Add the first delay parameter and the second delay parameter The maximum value in the time frame is used as the unified target time. ; The future vehicle motion state prediction module is configured to output the visual prediction state corresponding to the prediction time domain based on the current driving operation input and the vehicle dynamics model. and platform prediction status ; The visual rendering target generation module is configured to use the same vehicle dynamics model and the same current control input, with the unified target time. Based on the aforementioned visual prediction state Generate visual rendering target data; The platform control target generation module is configured to use the same vehicle dynamics model and the same current control input, with the unified target time. Based on the platform's predicted state, the baseline is used. Generate platform control target data, wherein the platform control target data serves as the theoretical somatosensory target state, including the platform's desired linear acceleration and desired angular velocity, which are determined by the platform's predicted state. The platform feedforward control quantity is obtained after washing. Then obtained through platform kinematic mapping; The platform execution-side synchronization error compensation module is configured to collect the linear acceleration and angular velocity output by the platform's IMU in real time, and compare them with the theoretical tactile target state to obtain the synchronization error vector. The synchronization error vector The outer loop feedback correction quantity is generated through proportional-integral conversion. The outer loop feedback correction amount Superimposed on the platform feedforward control quantity To form integrated control commands This drives the platform's mechanical execution link output to synchronize with the visual image, enabling haptic motion.

10. The VR driving simulation vision and motion-sensory synchronous control device according to claim 9, characterized in that, It also includes a frequency domain reallocation processing module, which is configured to decompose the theoretical haptic target state into low-frequency components when the platform approaches its physical limits. and high frequency components , in mapping ratio r Control the low-frequency component The mapping ratio is obtained by replacing direct translation with a tilted posture. r The proportion of constraints as the characterization platform approaches its physical limits α The low-frequency component changes continuously. The remaining part and the high-frequency component Execute within the remaining range.

Images