A car driving simulator in-vehicle simulation system

Abstract

This invention relates to the field of digital modeling technology and discloses a vehicle-machine simulation system for a car driving simulator. The system constructs a driving interaction state by collecting data on driving dynamics, driver eye movement characteristics, and cabin ergonomics assessment results. Based on a multi-objective reinforcement decision-making mechanism, it jointly optimizes the vehicle-machine interaction strategy, visual cue strategy, and steering wheel and pedal feedback parameters. Simultaneously, it uses a driving interaction neural control barrier function to perform safety constraint screening of vehicle-machine interaction actions. The system can dynamically adjust the central control display, head-up display, and feedback parameters according to cognitive load, vehicle stability, and ergonomic risk status, achieving coordinated control of driving behavior and vehicle-machine interaction behavior, thus improving the realism, interaction safety, and training evaluation accuracy of the driving simulation process.

Description

Technical Field

[0001] This invention relates to the field of digital modeling technology, and in particular to a car driving simulator vehicle simulation system. Background Technology

[0002] With the development of intelligent cockpits, vehicle-to-everything (V2X) systems, and virtual driving simulation technology, car driving simulators have gradually evolved from traditional mechanical driving training equipment towards intelligence, digitalization, and interactivity. Existing intelligent driving simulation technologies typically simulate the driving environment through 3D scene rendering, vehicle dynamics modeling, V2X interface display, and steering wheel and pedal feedback devices, and combine some driving behavior recognition algorithms to analyze the driving operation process. However, most existing technologies only model vehicle motion states or basic interaction logic. Their vehicle-machine interaction strategies typically employ fixed interface layouts, preset prompt rules, or static feedback parameters, lacking dynamic joint optimization capabilities based on driver cognitive load, visual attention changes, and ergonomic adaptation. This makes it difficult to adaptively adjust the vehicle-machine interaction process in real time according to different driving scenarios, driving risk states, and individual driver differences. At the same time, existing intelligent systems usually treat driving behavior analysis, eye-tracking analysis, ergonomic assessment, and vehicle-machine feedback control as independent functional modules, lacking a unified risk constraint decision-making mechanism. This leads to problems such as decreased road gaze ratio, increased vehicle-machine interaction interference, uncoordinated driving posture, and decreased vehicle control stability in complex road environments, frequent vehicle-machine operations, or high cognitive load states. It is difficult to effectively constrain high-risk interactive actions and achieve coordinated control between vehicle-machine display strategies, steering wheel force feedback, hydraulic pedal feedback, and head-up display prompts, thereby reducing the realism, interaction safety, and training and evaluation accuracy of the driving simulation process. Summary of the Invention

[0003] This invention aims to address the lack of a collaborative modeling mechanism for driving cognitive state, ergonomic state, and vehicle-machine interaction behavior in existing intelligent vehicle driving simulation systems. This leads to fixed vehicle-machine interaction strategies, strong interaction interference, insufficient safety boundary constraints, and low simulation realism and evaluation accuracy. The invention proposes a vehicle-machine simulation system for a driving simulator. This system achieves dynamic collaborative optimization of vehicle-machine interaction strategies, visual cue strategies, and steering wheel and pedal feedback parameters by fusing and modeling driving dynamics, driver cognitive load, eye-tracking behavior characteristics, and cockpit ergonomic state. Compared to existing vehicle-machine interaction control methods based on fixed rules or single vehicle states, this invention introduces a multi-objective reinforcement decision-making mechanism based on the utility of a risk sensitivity index. This enables the system to dynamically adjust high-risk vehicle-machine interaction behaviors based on road gaze ratio, cognitive load level, vehicle stability, and ergonomic risk state, thereby improving interaction safety and driving immersion realism in complex driving scenarios. Simultaneously, this invention constructs a driving interaction neural control barrier function. Through safety boundary search, Jacobian boundary propagation, and Lie derivative verification, it performs forward invariant constraint screening on vehicle-machine interaction actions, ensuring that the vehicle-machine control strategy optimizes the interactive experience while keeping the driving state within a safe set. Furthermore, this invention combines driver eye movement temporal feature analysis and digital human inverse kinematics posture prediction to jointly evaluate vehicle-machine gaze behavior, driving posture coordination, pedal accessibility, and visual field occlusion. Based on the evaluation results, it adjusts the central control display, head-up display, steering wheel force feedback, and hydraulic pedal feedback in a coordinated manner, thereby improving the realism, interaction consistency, and training evaluation accuracy of the driving simulation training process.

[0004] This invention provides a vehicle driving simulator system, comprising a hardware device layer and a functional logic layer. The hardware device layer includes a driving simulation cockpit, a hydraulic three-pedal mechanism, a steering wheel force feedback mechanism, a driving scene rendering server, a vehicle-machine interaction controller, a driver status acquisition device, and a cockpit ergonomics assessment server. The driving simulation cockpit is equipped with a central control screen, an instrument display screen, a head-up display unit, and pedal vibration feedback components. The hydraulic three-pedal mechanism, including a brake mechanism, a clutch mechanism, and an accelerator mechanism, is located at the bottom of the driving simulation cockpit and is used to collect brake pedal pressure, brake pedal travel, clutch pedal travel, and accelerator pedal opening, providing the driver with hydraulic pressure feedback, spring resistance feedback, and pedal vibration feedback. The functional logic layer includes a driving task analysis module, a 3D scene construction module, a scene rendering module, a driving state analysis module, an ergonomics assessment module, a vehicle-machine interaction decision-making module, and a feedback control module.

[0005] The driving task parsing module receives driving training tasks, vehicle types, road types, traffic density, weather conditions, vehicle-machine interaction tasks, and driver training objectives from the driving scene rendering server. It parses the driving training tasks, vehicle types, road types, traffic density, weather conditions, vehicle-machine interaction tasks, and driver training objectives into driving task semantic information, and generates a driving scene semantic map including road objects, traffic participation objects, lane structure objects, in-vehicle and vehicle-machine objects, and interactive prompt objects based on the driving task semantic information.

[0006] The 3D scene construction module constructs an initial layout of the 3D driving environment based on the semantic graph of the driving scene. The initial layout of the 3D driving environment includes road topology, lane lines, traffic signals, surrounding vehicles, obstacles, navigation path, cockpit view position, vehicle display area and interactive objects for the driver. Based on geometric drivability constraints, traffic semantic constraints, driving task constraints and vehicle-machine interaction reachability constraints, the module optimizes the spatial layout of the initial layout of the 3D driving environment to generate an interactive 3D driving scene.

[0007] The scene rendering module uses the driving scene rendering server to synchronously render the interactive 3D driving scene and obtain the initial rendering result of the vehicle simulation.

[0008] The driving dynamics module allows the driver to perform driving operations and vehicle-machine interaction operations based on the initial rendering results of the vehicle-machine simulation. During the driver's actions of pressing the brake pedal, clutch pedal, accelerator pedal, steering wheel, and engaging in vehicle-machine interaction, the vehicle-machine interaction controller simultaneously collects data on brake pedal pressure, brake pedal travel, clutch pedal travel, accelerator pedal opening, steering wheel angle, steering wheel angular velocity, steering wheel feedback torque, pedal vibration status, and driver-vehicle-machine operation events, generating driving operation timing data in chronological order. Based on this timing data, the module calculates vehicle longitudinal acceleration, braking force, engine speed, clutch engagement state, throttle output ratio, vehicle lateral deviation, heading error, and lane keeping error, generating vehicle dynamics state data.

[0009] The driver assessment module collects driver eye movement data by a driver state acquisition device set in the driving simulation cockpit during the process of the driver performing driving operations and vehicle-machine interaction operations based on the initial rendering results of the vehicle-machine simulation. The driver eye movement data is converted into a driver eye movement temporal feature table. The driver eye movement temporal feature table is then used to perform task guidance interpretation and user individual baseline calibration to generate driver cognitive load assessment results.

[0010] The ergonomics assessment module, the cockpit ergonomics assessment server, constructs a digital human body model of the driver based on the driver's height, sitting posture, seat position, steering wheel position, pedal position, vehicle screen position, instrument display position, and head-up display position. Based on the driver's digital human body model, it performs inverse kinematics posture prediction on the driver's limb movement posture during driving operations and vehicle-machine interaction operations based on the initial rendering results of the vehicle simulation. Combined with pedal operation reachability domain analysis and field of view occlusion analysis, it generates cockpit ergonomics assessment results.

[0011] The enhanced decision optimization module uses a multi-objective constraint enhanced decision model to optimize vehicle-machine interaction strategies and make joint decisions on feedback parameters based on vehicle dynamics state data, driving operation timing data, driver cognitive load assessment results, and cabin ergonomics assessment results, thereby generating vehicle-machine simulation control strategies.

[0012] The linkage evaluation module dynamically updates the central control vehicle interface status, instrument display status, head-up display information, vehicle display status, and road view image in the initial rendering result of the vehicle simulation according to the vehicle simulation control strategy. It also controls the central control vehicle screen, instrument display screen, head-up display unit, steering wheel force feedback mechanism, hydraulic three-pedal mechanism, and pedal vibration feedback component to perform linkage feedback. At the same time, it records the dynamically updated vehicle simulation rendering result, driving operation timing data, vehicle dynamics status data, driver cognitive load assessment result, cabin ergonomics assessment result, and vehicle simulation control strategy to generate a driving simulation record. The driving simulation records are segmented to obtain segmented records for the starting phase, acceleration phase, gear shifting phase, following phase, curve phase, intersection phase, vehicle-machine operation phase, and braking and stopping phase. The stability of pedal control, steering wheel control, lane keeping error, braking reaction time, vehicle-machine gaze ratio, cognitive load level change, ergonomic risk score, and vehicle-machine prompt response time are statistically analyzed for each segment. A driving training evaluation report is generated based on the statistical results. This report includes a driving operation score, a vehicle-machine interaction load score, a pedal operation score, an attention stability score, a cockpit adaptation score, and improvement suggestions.

[0013] By adopting the above solution, the beneficial effects achieved by the present invention are as follows:

[0014] Compared with existing car driving simulation systems, this invention achieves collaborative analysis and dynamic linkage control between vehicle-machine interaction behavior and driving behavior by uniformly integrating and modeling driving dynamics, driver cognitive load, eye movement behavior characteristics, and cabin ergonomics. This enhances the adaptive adjustment capability of vehicle-machine interaction strategies during driving simulation and solves the problem of fixed vehicle-machine interaction rules in existing technologies, which cannot be dynamically adjusted according to the driver's real-time state. At the same time, by jointly analyzing the road gaze ratio, vehicle-machine gaze ratio, cognitive load level, and vehicle stability state, the vehicle-machine simulation system can dynamically adjust the central control display level, prompt intensity, and interaction complexity according to different driving scenarios. This reduces the interference of vehicle-machine interaction on driving behavior and enhances the realism, immersion, and interaction consistency of the driving simulation process in complex driving scenarios.

[0015] This invention introduces a multi-objective enhanced decision-making mechanism based on the utility of a risk sensitivity index, achieving joint optimization of vehicle-machine interaction strategies, steering wheel force feedback, and hydraulic pedal feedback parameters. This enhances the dynamic response capability of the vehicle-machine simulation system to high-risk driving states and solves the problem in existing intelligent driving simulation systems that rely solely on a single vehicle state for interactive control and struggle to identify high-risk interactive behaviors. Furthermore, this invention combines road gaze risk, braking response risk, vehicle steering stability risk, cognitive load risk, and ergonomic risk to dynamically penalize and adjust strategies for high-risk vehicle-machine interaction behaviors. This enables the vehicle-machine simulation system to proactively reduce non-critical interactive interference when the driver's cognitive load increases or vehicle stability decreases, thereby enhancing safety, stability, and the ability to simulate risky scenarios during driving training.

[0016] Furthermore, this invention constructs a driving interaction neural control barrier function and combines it with safety boundary candidate sub-region search, Jacobian boundary propagation, and Lie derivative upper bound verification to achieve forward invariant safety constraint screening for vehicle-machine interaction actions. This enhances the safety boundary control capability of the vehicle-machine simulation system under complex interaction states and solves the problem that existing driving simulation systems lack dynamic safety constraints and are prone to driving state overstepping due to vehicle-machine interaction. At the same time, this invention further combines driver eye movement temporal feature analysis and digital human inverse kinematics posture prediction to evaluate the driving posture coordination state, pedal accessibility, and field of vision occlusion state in real time. Based on the evaluation results, it dynamically adjusts the steering wheel force feedback, hydraulic pedal feedback, and head-up display prompt strategies, thereby enhancing the ergonomic adaptability, operational comfort, and training evaluation accuracy during driving simulation training. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the modules of an in-vehicle driving simulator system proposed in this invention;

[0018] Figure 2 This is a schematic diagram of the safety boundary of the driving interaction neural control barrier function proposed in Embodiment 5 of the present invention; wherein, the horizontal axis represents the lateral offset of the vehicle. The vertical axis represents the degree of lateral deviation of the vehicle from the lane centerline; the vertical axis represents the vehicle-to-infrastructure gaze ratio, representing the proportion of time the driver gazes at the central control area of ​​the vehicle during the current driving cycle; the color bar on the right represents the driving interaction safety barrier value. The size of the area is indicated by the color, where a color closer to green indicates a safer current driving interaction state, and a color closer to red indicates a current driving interaction state closer to a danger zone; the green area in the image represents the safe zone, corresponding to... The yellow area represents the driving interaction state satisfying the forward invariance constraint; the yellow area represents the safety boundary region, corresponding to... The red area represents the driving interaction state approaching the safety boundary; the red area represents the unsafe area, corresponding to... This is used to characterize the risk of exceeding the limits in the driving interaction state; in the figure , , Indicates the target candidate action that satisfies the safety constraints; , , The blue dashed line represents the candidate action that was rejected because it did not meet the safety constraints; the blue dashed line represents the safety boundary curve. The green curve represents the boundary of the safe zone. The red curve represents the boundary of the unsafe zone. . Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] Example 1, according to Figure 1This invention provides a vehicle driving simulator system, comprising a hardware device layer and a functional logic layer. The hardware device layer includes a driving simulation cockpit, a hydraulic three-pedal mechanism, a steering wheel force feedback mechanism, a driving scene rendering server, a vehicle-machine interaction controller, a driver status acquisition device, and a cockpit ergonomics assessment server. The driving simulation cockpit is equipped with a central control screen, an instrument display screen, a head-up display unit, and pedal vibration feedback components. The hydraulic three-pedal mechanism, comprising a brake mechanism, a clutch mechanism, and an accelerator mechanism, is located at the bottom of the driving simulation cockpit and is used to acquire brake pedal pressure, brake pedal travel, clutch pedal travel, and accelerator pedal opening, providing the driver with hydraulic pressure feedback, spring resistance feedback, and pedal vibration feedback. The functional logic layer includes a driving task analysis module, a 3D scene construction module, a scene rendering module, a driving state analysis module, an ergonomics assessment module, a vehicle-machine interaction decision-making module, and a feedback control module.

[0021] In this embodiment, the driving simulation cockpit adopts a closed single-person cockpit structure. The seat height adjustment range is 180mm, and the steering wheel supports 80mm of fore-and-aft adjustment and 35mm of height adjustment. The central control screen is 15.6 inches in size with a resolution of 2560×1440. The instrument display screen is 12.3 inches in size. The head-up display unit has a projection distance of 2.8m. The maximum output torque of the steering wheel force feedback mechanism is 8.5N·m. In the hydraulic three-pedal mechanism, the maximum feedback pressure of the brake pedal is 420N, the maximum damping force of the clutch pedal is 210N, and the rebound resistance of the accelerator pedal ranges from 15N to 65N.

[0022] The driving task parsing module receives driving training tasks, vehicle types, road types, traffic density, weather conditions, vehicle-machine interaction tasks, and driver training objectives from the driving scene rendering server. It parses the driving training tasks, vehicle types, road types, traffic density, weather conditions, vehicle-machine interaction tasks, and driver training objectives into driving task semantic information, and generates a driving scene semantic map including road objects, traffic participation objects, lane structure objects, in-vehicle and vehicle-machine objects, and interactive prompt objects based on the driving task semantic information.

[0023] In this embodiment, the driving training task is set as follows:

[0024] The total length of the driving route is 6.8km; the road types include urban arterial roads, two-way four-lane roads, elevated ramps and complex intersections; the traffic density is set to 58 vehicles / km; the weather conditions are set to moderate rain; the training objectives include following other vehicles, keeping up on curves, navigating complex intersections, and vehicle-to-vehicle navigation interaction; the vehicle-to-vehicle interaction tasks include modifying the navigation destination, adjusting the air conditioning temperature, and handling incoming call notifications.

[0025] Extract the following semantic information for driving tasks:

[0026] Task start coordinates: (31.2315, 121.4721); Task end coordinates: (31.2268, 121.4897); Road scenario type: Urban peak traffic congestion scenario; Driving behavior type: Low-speed following + continuous lane changing + ramp merging; Traffic event type: Sudden deceleration of the vehicle in front, pedestrian crossing, forced lane change by the vehicle on the right; Vehicle-machine interaction requirements: Navigation replanning + air conditioning adjustment while driving.

[0027] The 3D scene construction module constructs an initial layout of the 3D driving environment based on the semantic graph of the driving scene. The initial layout of the 3D driving environment includes road topology, lane lines, traffic signals, surrounding vehicles, obstacles, navigation path, cockpit view position, vehicle display area and interactive objects for the driver. Based on geometric drivability constraints, traffic semantic constraints, driving task constraints and vehicle-machine interaction reachability constraints, the module optimizes the spatial layout of the initial layout of the 3D driving environment to generate an interactive 3D driving scene.

[0028] The scene rendering module uses a driving scene rendering server to synchronously render the interactive 3D driving scene and obtain the initial rendering result of the vehicle simulation. The initial rendering result of the vehicle simulation includes the road view, vehicle display status, instrument display status, central control vehicle interface status, and head-up display information.

[0029] The driving dynamics module allows the driver to perform driving operations and vehicle-machine interaction operations based on the initial rendering results of the vehicle-machine simulation. During the driver's actions of pressing the brake pedal, clutch pedal, accelerator pedal, steering wheel, and engaging in vehicle-machine interaction, the vehicle-machine interaction controller simultaneously collects data on brake pedal pressure, brake pedal travel, clutch pedal travel, accelerator pedal opening, steering wheel angle, steering wheel angular velocity, steering wheel feedback torque, pedal vibration status, and driver-vehicle-machine operation events, generating driving operation timing data in chronological order. Based on this timing data, the module calculates vehicle longitudinal acceleration, braking force, engine speed, clutch engagement state, throttle output ratio, vehicle lateral deviation, heading error, and lane keeping error, generating vehicle dynamics state data.

[0030] The driver assessment module collects driver eye-tracking data by a driver state acquisition device located in the driving simulation cockpit during the driver's driving and vehicle-machine interaction operations based on the initial rendering results of the vehicle-machine simulation. The driver eye-tracking data is then converted into a driver eye-tracking temporal feature table. Task guidance interpretation and user individual baseline calibration are performed on the driver eye-tracking temporal feature table to generate a driver cognitive load assessment result. The driver cognitive load assessment result includes cognitive load level, degree of attention distraction, degree of vehicle-machine interaction interference, and driver state interpretation information.

[0031] The ergonomics assessment module, via a cockpit ergonomics assessment server, constructs a digital human body model of the driver based on the driver's height, posture, seat position, steering wheel position, pedal position, vehicle infotainment screen position, instrument display position, and head-up display position. Based on this model, the module performs inverse kinematics posture prediction on the driver's limb movements during driving and vehicle-to-vehicle interaction operations, using the initial rendering results from the vehicle-to-vehicle simulation. This prediction is combined with pedal operation reachability analysis and field-of-view occlusion analysis to generate cockpit ergonomics assessment results. These results include driving comfort, vehicle-to-vehicle interaction accessibility, field-of-view occlusion level, and driving posture coordination.

[0032] The enhanced decision optimization module uses a multi-objective constraint enhanced decision model to optimize vehicle-machine interaction strategies and make joint decisions on feedback parameters based on vehicle dynamics state data, driving operation timing data, driver cognitive load assessment results, and cabin ergonomics assessment results, thereby generating vehicle-machine simulation control strategies.

[0033] In this embodiment, the enhanced decision optimization module performs vehicle-machine interaction strategy optimization with a driving simulation decision cycle of 100ms. At the current complex intersection stage, the driving interaction state constructed by the system includes: vehicle speed: 38km / h; lateral offset: 0.26m; steering wheel angular velocity: 142° / s; braking reaction time: 0.73s; vehicle-machine gaze ratio: 26%; cognitive load level: Level III; ergonomic risk score: 0.28.

[0034] The system generates 12 candidate vehicle-machine control actions within the current decision cycle and performs forward invariance constraint screening through a neural control barrier function. Among them, 3 actions are eliminated because they may cause the vehicle to deviate beyond the boundary; 2 actions are eliminated because the vehicle-machine gaze time exceeds the limit; and finally, 7 executable actions are retained.

[0035] Subsequently, the enhanced decision optimization module executes a Bellman-style policy value recursion, which converges after 14 rounds of fixed-point iterations; the final stable policy value corresponding to the target action is determined to be 8.74; the corresponding generated vehicle-to-everything (V2X) simulation control strategy is as follows:

[0036] 1. Central control vehicle interface display strategy: The navigation interface is switched to simplified mode; the entertainment function entry is hidden; the air conditioning control interface is compressed from a three-level menu to a single-level sliding control; all non-critical notifications are collapsed; the information hierarchy of the main vehicle interface is reduced from 5 layers to 2 layers.

[0037] 2. Instrument display strategy: The vehicle speed display priority is adjusted to the highest level; the navigation turn prompt is moved to the central area; the engine speed display is reduced by 18%; non-critical energy consumption information is paused; the collision risk warning uses a dynamic red flashing.

[0038] 3. HUD prompt strategy: Navigation arrow transparency increased to 92%; lane keeping prompt width increased by 15%; forward distance prompt refresh frequency increased to 20Hz; risk warning early trigger time adjusted from 1.2s to 2.0s.

[0039] 4. Steering wheel force feedback control parameters: return torque increased from 2.8 N·m to 4.1 N·m; damping torque increased from 1.3 N·m to 2.2 N·m; peak correction torque limited to 5.4 N·m.

[0040] 5. Hydraulic three-pedal feedback mapping parameters: Brake pressure mapping gain increased by 12%; clutch engagement damping decreased by 9%; sensitivity in the first 30% range of throttle opening decreased by 15%.

[0041] 6. Pedal vibration feedback parameters: Vibration frequency is 38Hz when ABS is triggered; vibration frequency is 24Hz when there is a risk of slippery road surface; vibration duration is 320ms when there is a collision risk; vibration amplitude is increased by 20% for emergency braking warning.

[0042] The linkage evaluation module dynamically updates the central control vehicle interface status, instrument display status, head-up display information, vehicle display status, and road view image in the initial rendering result of the vehicle simulation according to the vehicle simulation control strategy. It also controls the central control vehicle screen, instrument display screen, head-up display unit, steering wheel force feedback mechanism, hydraulic three-pedal mechanism, and pedal vibration feedback component to perform linkage feedback. At the same time, it records the dynamically updated vehicle simulation rendering result, driving operation timing data, vehicle dynamics status data, driver cognitive load assessment result, cabin ergonomics assessment result, and vehicle simulation control strategy to generate a driving simulation record. The driving simulation records are segmented to obtain segmented records for the starting phase, acceleration phase, gear shifting phase, following phase, curve phase, intersection phase, vehicle-machine operation phase, and braking and stopping phase. The stability of pedal control, steering wheel control, lane keeping error, braking reaction time, vehicle-machine gaze ratio, cognitive load level change, ergonomic risk score, and vehicle-machine prompt response time are statistically analyzed for each segment. A driving training evaluation report is generated based on the statistical results. This report includes a driving operation score, a vehicle-machine interaction load score, a pedal operation score, an attention stability score, a cockpit adaptation score, and improvement suggestions.

[0043] Example 2, based on Example 1, describes how the driver's eye movement temporal feature table is constructed:

[0044] The driver status acquisition device collects the driver's left eye gaze direction, right eye gaze direction, pupil diameter, blink events, fixation point coordinates, and head posture at a preset sampling frequency. The acquired eye movement data undergoes time synchronization, outlier removal, blink loss compensation, and coordinate mapping processing to obtain standardized eye movement data. The standardized eye movement data is segmented according to sliding time windows. Within each time window, the fixation duration, saccade duration, saccade amplitude, fixation ratio, saccade ratio, blink count, mean pupil size, and pupil change trend are calculated. The eye movement features corresponding to each time window are organized chronologically into a driver eye movement temporal feature table, which simultaneously characterizes the driver's fixation stability, visual search intensity, blink changes, and cognitive load change trends.

[0045] Example 3, based on Example 2, in which the individual user baseline calibration includes:

[0046] In the initial stage of driving simulation, eye movement data of drivers are collected under low-load cruise, medium-load following, and high-load complex intersection tasks. The driver's pupil baseline, pupil sensitivity, blink baseline, and gaze stability baseline are calculated. Based on the pupil baseline, pupil sensitivity, blink baseline, and gaze stability baseline, the driver is classified into high-reaction, low-reaction, or unstable gaze driver profiles.

[0047] During real-time cognitive load assessment, the threshold for interpreting eye movement features is adjusted based on the driver profile. When the driver is low-reactivity type, the load judgment threshold corresponding to pupil changes and fixation changes is reduced. When the driver is unstable fixation type, the weight of the impact of a single saccade abnormality on attentional distraction is reduced, and the judgment weight of continuous abnormal segments is increased. When the driver is high-reactivity type, the judgment threshold for instantaneous pupil changes on high cognitive load is increased, thereby reducing misjudgments caused by individual differences in eye movements.

[0048] Example 4, based on Example 3, describes how the driver's digital human body model is constructed:

[0049] The cockpit ergonomics assessment server receives driver height, weight, seat height, shoulder width, arm length, leg length, and driving posture calibration data. Based on these human dimensional parameters, it generates a parametric human skeleton model including the pelvis, torso, neck, shoulders, elbows, wrists, hips, knees, and ankles. Based on seat position, backrest angle, steering wheel position, pedal position, and infotainment screen position, it determines the driver's hand, foot, and line-of-sight target points. Through inverse kinematics, it solves for the driver's joint angles during steering wheel operation, braking operation, clutch operation, accelerator operation, and infotainment screen touch operation, generating a driving posture sequence. Based on this sequence, it calculates the pedal operation reachability, steering wheel operation comfort, infotainment screen touch accessibility, instrument panel visibility, head-up display visibility, and cockpit occlusion relationships, generating the cockpit ergonomics assessment results.

[0050] Example 5, according to Figure 2 This embodiment is based on Embodiment 4. In this embodiment, based on vehicle dynamics state data, driving operation timing data, driver cognitive load assessment results, and cabin ergonomics assessment results, a multi-objective constraint reinforcement decision-making model is used to optimize the vehicle-machine interaction strategy and jointly decide on feedback parameters to generate a vehicle-machine simulation control strategy. The specific steps include:

[0051] Step S1: The vehicle-machine interaction controller uses the current driving simulation decision cycle as the time index to synchronize and align vehicle dynamics state data, driving operation timing data, driver cognitive load assessment results, and cabin ergonomics assessment results. It extracts driving interaction state features, including vehicle speed, longitudinal acceleration, lateral deviation, heading error, lane keeping error, braking reaction time, steering wheel angle, steering wheel angular velocity, brake pedal pressure, accelerator pedal opening, road gaze ratio, vehicle-machine gaze ratio, cognitive load level, vehicle-machine interaction interference level, pedal accessibility score, steering wheel operation comfort score, vehicle-machine touch accessibility score, and occlusion risk score, forming the driving interaction state. Candidate actions are defined, and a driving interaction neural control barrier function is constructed based on the driving interaction state. The candidate actions are then screened using forward invariance constraints through safety boundary candidate sub-region search, Jacobian boundary propagation, and Lie derivative upper bound verification, resulting in the current driving interaction state. The set of executable actions in the pre-driving interaction state; combining the driving interaction state with the executable actions in the set of executable actions to form an executable state-action pair; wherein, the pedal accessibility score is used to characterize the joint range of motion and pedal stability of the driver's foot when switching between the brake pedal, clutch pedal and accelerator pedal; the steering wheel operation comfort score is used to characterize the degree of deviation of the shoulder, elbow and wrist joint angles of the driver's hands when holding the steering wheel from the comfortable neutral posture; the vehicle touch screen accessibility score is used to characterize the upper limb extension distance and torso deflection degree of the driver when touching different areas of the central control vehicle screen while maintaining the driving posture; the field of vision visibility score is used to characterize the visibility of the instrument display, head-up display unit, road center area and side rear view areas in the driver's field of vision; the occlusion risk score is used to characterize the degree of occlusion of key visual areas by the steering wheel, A-pillar, vehicle screen bezel, seat posture and driver head posture.

[0052] The driving interaction neural control barrier function is constructed using a feedforward neural network with a smooth nonlinear activation function. The network input layer receives the driving interaction state, the hidden layer uses a tanh activation function, and the output layer outputs a scalar form of the driving interaction safety barrier value. The driving interaction safety barrier value is used to characterize the degree of deviation of the current driving interaction state from the driving interaction safety set. When the driving interaction safety barrier value is on the safe side, the driving interaction state is determined to be within the safe set. When the driving interaction safety barrier value is close to the boundary, the driving interaction state is determined to be close to the safe boundary. When the driving interaction safety barrier value is on the unsafe side, the driving interaction state is determined to have a risk of exceeding the boundary.

[0053] Step S2: The vehicle-machine interaction controller calculates the immediate reward for each executable state-action pair, performs a risk sensitivity index utility transformation on the immediate reward of each executable state-action pair, and constructs an expected index utility evaluation quantity. Specifically, when the action corresponding to the executable state-action pair leads to a decrease in road gaze ratio, an increase in vehicle-machine gaze ratio, an increase in braking reaction time, a decrease in steering wheel control stability, an increase in cognitive load level, an increase in vehicle-machine interaction interference level, or an increase in ergonomic risk score, the vehicle-machine interaction controller amplifies the corresponding negative reward through index utility transformation, so that vehicle-machine interference states with low occurrence frequency but high degree of danger receive higher penalties in the strategy value calculation; when the action corresponding to the executable state-action pair can reduce vehicle-machine interaction steps, increase road gaze ratio, reduce cognitive load level, or improve feedback consistency, the vehicle-machine interaction controller reduces its risk penalty weight.

[0054] The expected index utility evaluation metric is generated as follows:

[0055] ;

[0056] in, Indicates the current driving interaction status; Indicates the current candidate vehicle control action; Indicates the first State-action pair within a driving simulation decision cycle The corresponding expected index utility evaluation quantity; This indicates a risk-sensitive parameter used to adjust the sensitivity of the vehicle-machine interface controller to high-risk driving interaction states; Represents the immediate feedback of state-action pairs; This indicates the risk items related to vehicle-machine interaction; This represents the adjustment coefficient for vehicle-machine interaction risk items;

[0057] Represented as:

[0058] ;

[0059] in, This indicates the visual attention risk caused by a decrease in the proportion of road-focused attention and an increase in the proportion of vehicle-mounted camera-focused attention; This indicates the risk to braking response caused by increased braking reaction time and sudden changes in brake pedal pressure. This indicates steering stability risks caused by abnormal steering wheel angular velocity, increased lateral offset, and increased heading error. This indicates the risk of cognitive load caused by increased cognitive load level and increased attentional distraction; This indicates ergonomic risks caused by reduced pedal accessibility, reduced in-vehicle touchscreen accessibility, and increased risk of obstruction. This indicates the risk of interference with vehicle-machine interaction caused by an increase in the number of display layers, touch steps, or the frequency of non-critical prompts in the vehicle-machine interface. , , , , , These represent the weighting coefficients of each risk item;

[0060] Risk-sensitive parameters It can be represented as:

[0061] ;

[0062] in, Indicates the basic risk-sensitive parameter; The normalized value of cognitive load level is obtained by weighting and normalizing the driver's cognitive load level, pupil change, scan rate, blink rate change, and attentional distraction level. The normalized risk value of the proportion of vehicle-to-machine gaze is obtained by weighting and normalizing the proportion of vehicle-to-machine gaze duration, the decrease in the proportion of road gaze, the continuous duration of vehicle-to-machine gaze, and the number of times the gaze shifts from the road area to the vehicle-to-machine area. The normalized value of vehicle dynamics risk is represented by the weighted normalization of changes in vehicle lateral deviation, heading error, lane keeping error, braking reaction time, steering wheel angular velocity, and longitudinal acceleration. The ergonomic risk normalization value is obtained by weighted normalization of pedal accessibility score, steering wheel operation comfort score, vehicle touch accessibility score, field of vision visibility score and obstruction risk score. , , , These represent risk sensitivity adjustment coefficients;

[0063] Step S3: The vehicle-machine interaction controller performs Bellman-style policy value recursion based on the expected index utility evaluation quantity to obtain the policy value of the current executable state-action pair; it updates the policy value at fixed points according to a random approximation step size, and compares the policy value obtained in the current iteration with the policy value in the previous iteration to obtain the policy value residual; when the policy value residual is greater than a preset residual threshold, the policy value update continues; when the policy value residual is continuously less than the preset residual threshold, the current policy value is determined as the fixed point approximation result, which is used as the stable policy value corresponding to each executable action; Bellman-style policy value recursion is performed, specifically: predicting the next driving interaction state based on the current driving interaction state and the current executable action, and reading the policy value estimate corresponding to each executable action in the next driving interaction state; recursively fusing the expected index utility evaluation quantity of the current executable state-action pair, the optimal policy value estimate in the next driving interaction state, and the discount factor to obtain the policy value of the current executable state-action pair;

[0064] Perform fixed-point iterative updates according to a random approximation step size, expressed as:

[0065] ;

[0066] in, Indicates the first State-action pairs in the next fixed-point iteration Strategic value estimation; Indicates the first State-action pairs in the next fixed-point iteration Strategic value estimation; This represents the random approximation step size, used to control the update magnitude of the current recursive value to the historical strategy value; Represents the current state-action pair The corresponding target strategy value is generated by fusing the immediate risk-sensitive benefit in the current driving interaction state with the discounted optimal strategy value estimate in the next driving interaction state through Bellman-type strategy value recursion. This is used for fixed-point iterative updates of the execution state-action strategy value.

[0067] Step S4: Based on the stable strategy value corresponding to each executable action, a greedy strategy selection is performed, and the executable action that satisfies the optimal strategy value condition is determined as the target action; the target action is converted into a vehicle-machine simulation control strategy, which includes a central control vehicle interface display strategy, an instrument display strategy, a head-up display prompt strategy, steering wheel force feedback control parameters, hydraulic three-pedal feedback mapping parameters, and pedal vibration feedback parameters; wherein, the central control vehicle interface display strategy includes information display hierarchy, number of interaction entry points, and non-critical function suppression status; the instrument display strategy includes the display priority of vehicle speed, engine speed, gear, braking status, throttle status, clutch status, and warning status; the head-up display prompt strategy includes the display intensity of navigation arrows, lane keeping prompts, forward distance prompts, and risk warnings; the steering wheel force feedback control parameters include return torque, damping torque, and correction torque; the hydraulic three-pedal feedback mapping parameters include brake pressure mapping, clutch engagement mapping, and throttle opening mapping; and the pedal vibration feedback parameters include vibration trigger time, vibration amplitude, and vibration duration.

[0068] Example 6, based on Example 5, utilizes safety boundary candidate sub-region search, Jacobian boundary propagation, and Lie derivative upper bound verification to perform forward invariance constraint screening on candidate actions, obtaining the set of executable actions in the current driving interaction state. The specific steps include:

[0069] Step E1: Construct a driving interaction state space based on the driving interaction state; partition the driving interaction state space to form multiple axis-aligned hyperrectangular sub-regions; extract all vertex states of each sub-region and input each vertex state into the driving interaction neural control barrier function to obtain the driving interaction safety barrier value corresponding to each vertex; when the safety barrier values ​​of all vertices within the same axis-aligned hyperrectangular sub-region are all on the safe side, the sub-region as a whole is determined to be within the driving interaction safety set; when the safety barrier values ​​of all vertices within the same axis-aligned hyperrectangular sub-region are all on the unsafe side, the sub-region as a whole is determined to be within the driving interaction safety set. Externally; when both safe-side vertices and non-safe-side vertices exist simultaneously within the same axis-aligned super-rectangular sub-region, or when the safety barrier value of at least one vertex is close to the safety boundary tolerance, the sub-region is determined to contain or be adjacent to the driving interaction safety boundary, and is identified as a candidate sub-region for the safety boundary; the candidate sub-region for the safety boundary is used as the object of subsequent safety verification; when the next driving interaction state corresponding to the candidate action is far away from the candidate sub-region for the safety boundary, the candidate action is marked as a low-risk candidate action; when the next driving interaction state corresponding to the candidate action enters or crosses the candidate sub-region for the safety boundary, the candidate action is marked as an action to be verified;

[0070] Step E2: For the candidate sub-regions of the safety boundary, perform forward interval propagation on the driving interaction neural control barrier function to obtain the interval boundaries of the pre-activation variables of each network layer; based on the interval boundaries, use the analytical property of the derivative of the smooth nonlinear activation function to calculate the derivative boundary of the activation function, and determine the lower and upper bounds of the activation derivative matrix of each network layer; wherein, when the driving interaction neural control barrier function adopts the tanh activation function, the vehicle-machine interaction controller uses the analytical property that the derivative of the tanh function is symmetric about the zero point and obtains the maximum value at the zero point, and determines the derivative boundary of the activation function according to whether the interval of the pre-activation variable crosses the zero point and the derivative value corresponding to the endpoint of the interval; when the interval of the pre-activation variable crosses the zero point, the derivative value at the zero point is used as the upper bound of the derivative, and the smaller value of the derivative values ​​corresponding to the two endpoints of the interval is used as the lower bound of the derivative; when the interval of the pre-activation variable is located on the same side of the zero point, according to the monotonic change relationship of the derivative of the tanh function in the interval, the derivative values ​​corresponding to the two endpoints of the interval are used to determine the lower and upper bounds of the derivative, respectively;

[0071] When the driving interaction neural control barrier function adopts other smooth nonlinear activation functions, the vehicle-machine interaction controller determines the derivative stationary point according to the change law of the derivative of the smooth nonlinear activation function. The endpoints of the pre-activated variable interval and the derivative stationary points falling within the pre-activated variable interval are used as candidate points. The minimum and maximum values ​​of the derivative values ​​corresponding to each candidate point are selected as the lower bound and upper bound of the derivative of the activation function, respectively.

[0072] Subsequently, based on the lower and upper bounds of the activation derivative matrix, the activation derivative matrix boundary and network weight matrix of the current network layer are combined by backpropagation layer by layer to obtain the Jacobian lower and upper bounds of the driving interaction neural control barrier function with respect to the driving interaction state.

[0073] Step E3: For each action to be verified, construct a driving interaction state transition dynamics model under the action; using the midpoint of the candidate sub-region of the safety boundary as the Taylor expansion center, calculate the Taylor affine boundary of the driving interaction state transition dynamics model to obtain the lower and upper bounds of the state transition dynamics; based on the Jacobian lower and upper bounds, calculate the upper bound of the Lie derivative corresponding to the action to be verified; when the upper bound of the Lie derivative satisfies the forward invariance constraint of the neural control barrier function, it is determined that the action to be verified will not drive the driving interaction state out of the driving interaction safety set, and the action to be verified is retained; when the upper bound of the Lie derivative does not satisfy the forward invariance constraint, it is determined that the action to be verified has the risk of driving the driving interaction state out of the safety boundary, and the action to be verified is removed, thus obtaining the set of executable actions in the current driving interaction state.

[0074] Example 7, based on Example 6, in which the linkage feedback includes:

[0075] The central control screen displays navigation, vehicle status, entertainment information, air conditioning control, driving modes, and training task prompts according to the vehicle's simulated control strategy.

[0076] The instrument display shows vehicle speed, engine speed, gear, braking status, throttle status, clutch status, and warning status based on vehicle dynamics.

[0077] The head-up display unit displays lane keeping prompts, navigation arrows, forward distance prompts, and risk warnings based on road scene status, vehicle dynamics status data, and driver cognitive load assessment results.

[0078] The steering wheel force feedback mechanism outputs a return torque, damping torque, or correction torque based on road curvature, vehicle lateral deviation, road surface adhesion coefficient, and driving risk level.

[0079] The hydraulic three-pedal mechanism adjusts the braking pressure mapping, clutch engagement mapping, and throttle opening mapping based on vehicle dynamics data and driving task phases.

[0080] The pedal vibration feedback component outputs corresponding vibration feedback based on braking impact, engine speed changes, tire slippage, ABS activation, uneven road surface, and collision risk.

[0081] The present invention and its embodiments have been described above. This description is not restrictive. The accompanying drawings are only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this description and designs a similar structure and embodiment without departing from the spirit of the present invention, such design should fall within the protection scope of the present invention.

Claims

1. A car driving simulator in-vehicle simulation system, characterized in that: The system includes: The 3D scene construction module generates interactive 3D driving scenes. The scene rendering module synchronously renders the interactive 3D driving scene to obtain the initial rendering result of the vehicle simulation. The driving dynamics module executes driving operations and vehicle-machine interaction operations based on the initial rendering results of the vehicle-machine simulation; acquires driving operation timing data; and generates vehicle dynamics state data based on the driving operation timing data. The driver assessment module acquires the driver's eye movement time sequence feature table during the process of performing driving operations and vehicle-machine interaction operations based on the initial rendering results of the vehicle-machine simulation; it performs task guidance interpretation and user individual baseline calibration on the driver's eye movement time sequence feature table to generate driver cognitive load assessment results. The ergonomics assessment module generates cockpit ergonomics assessment results. The enhanced decision optimization module uses a multi-objective constraint enhanced decision model to optimize vehicle-machine interaction strategies and make joint decisions on feedback parameters based on vehicle dynamics state data, driving operation timing data, driver cognitive load assessment results, and cabin ergonomics assessment results, thereby generating vehicle-machine simulation control strategies. The linkage evaluation module dynamically updates the initial rendering results of the vehicle simulation according to the vehicle simulation control strategy, executes linkage feedback, and generates driving simulation records.

2. The vehicle driving simulator system according to claim 1, characterized in that: The process of using a multi-objective constraint-enhanced decision-making model to optimize vehicle-machine interaction strategies and jointly decide on feedback parameters to generate vehicle-machine simulation control strategies includes the following steps: Step S1: Synchronize and align vehicle dynamics state data, driving operation timing data, driver cognitive load assessment results, and cabin ergonomics assessment results to form a driving interaction state; define candidate actions, perform forward invariance constraint screening on candidate actions to obtain a set of executable actions; combine the driving interaction state with the executable actions in the set of executable actions to form an executable state-action pair; Step S2: Calculate the immediate reward for each executable state-action pair, perform a risk-sensitive index utility transformation on the immediate reward of each executable state-action pair, and construct the expected index utility evaluation quantity. Step S3: Based on the expected index utility evaluation, perform Bellman-type policy value recursion to obtain the policy value of the current executable state-action pair; perform fixed-point iterative updates on the policy value according to the random approximation step size to obtain the stable policy value corresponding to each executable action; Step S4: Perform greedy strategy selection based on the stable strategy value corresponding to each executable action to determine the target action; convert the target action into a vehicle-machine simulation control strategy.

3. The vehicle driving simulator system according to claim 2, characterized in that: Step S1 specifically includes: using the current driving simulation decision cycle as the time index, synchronizing and aligning vehicle dynamics state data, driving operation timing data, driver cognitive load assessment results, and cockpit ergonomics assessment results, extracting driving interaction state features, and forming driving interaction states; defining candidate actions, constructing a driving interaction neural control barrier function based on the driving interaction states, and using safety boundary candidate sub-region search, Jacobian boundary propagation, and Lie derivative upper bound verification to screen candidate actions with forward invariance constraints, obtaining a set of executable actions; and combining the driving interaction states with the executable actions in the set of executable actions to form executable state-action pairs.

4. The vehicle driving simulator system according to claim 2, characterized in that: Step S3 specifically includes: based on the expected index utility evaluation metric, performing Bellman-type policy value recursion to obtain the policy value of the current executable state-action pair; performing fixed-point iterative updates on the policy value according to a random approximation step size, and comparing the policy value obtained in the current iteration with the policy value in the previous iteration to obtain the policy value residual; when the policy value residual is greater than a preset residual threshold, continuing to perform policy value updates; when the policy value residual is continuously less than the preset residual threshold, determining the current policy value as the fixed-point approximation result, and using it as the stable policy value corresponding to each executable action.

5. The vehicle driving simulator system according to claim 3, characterized in that: The driving interaction neural control barrier function is constructed using a feedforward neural network with a smooth nonlinear activation function. The network input layer receives the driving interaction state, the hidden layer uses the tanh activation function, and the output layer outputs the driving interaction safety barrier value in scalar form.

6. The vehicle driving simulator system according to claim 5, characterized in that: The process of obtaining the set of executable actions includes the following steps: Step E1: Construct a driving interaction state space based on the driving interaction state; partition the driving interaction state space to form multiple axis-aligned hyperrectangular sub-regions; extract all vertex states of the sub-regions and input each vertex state into the driving interaction neural control barrier function to obtain the driving interaction safety barrier value corresponding to each vertex, and determine the candidate sub-regions of the safety boundary; use the candidate sub-regions of the safety boundary as the objects of subsequent safety verification and mark the actions to be verified. Step E2: For the candidate sub-regions of the safety boundary, perform forward interval propagation on the driving interaction neural control barrier function to obtain the interval boundaries of the pre-activation variables of each network layer; based on the interval boundaries, use the analytical properties of the derivative of the smooth nonlinear activation function to calculate the derivative boundary of the activation function, and determine the lower and upper bounds of the activation derivative matrix of each network layer; based on the lower and upper bounds of the activation derivative matrix, combine the activation derivative matrix boundary and the network weight matrix of the current network layer by backpropagation layer by layer to obtain the Jacobian lower and upper bounds of the driving interaction neural control barrier function with respect to the driving interaction state; Step E3: For each action to be verified, perform Taylor affine boundary calculation to obtain the lower bound and upper bound of the state transition dynamics; based on the Jacobian lower bound, Jacobian upper bound, lower bound and upper bound of the state transition dynamics, determine whether the action to be verified satisfies the forward invariance constraint, and obtain the set of executable actions.

7. The vehicle driving simulator system according to claim 6, characterized in that: Step E3 specifically includes: for each action to be verified, constructing a driving interaction state transition dynamics model under the action; using the midpoint of the candidate sub-region of the safety boundary as the Taylor expansion center, performing Taylor affine boundary calculation on the driving interaction state transition dynamics model to obtain the lower bound and upper bound of the state transition dynamics; based on the Jacobian lower bound, Jacobian upper bound, lower bound and upper bound of the state transition dynamics, calculating the upper bound of the Lie derivative corresponding to the action to be verified, and determining whether the action to be verified satisfies the forward invariance constraint to obtain the set of executable actions.

Images