Simulation driving system, method and equipment based on real vehicle condition, medium and product

Abstract

The invention discloses a simulation driving system, method and device based on real vehicle conditions, a medium and a product. The system is characterized in that a vehicle diagnosis interface and a vehicle bus are connected with a to-be-deployed vehicle, and a visual sensor is arranged on the to-be-deployed vehicle; the vehicle diagnosis interface is used for determining a data processing rule matched with the to-be-deployed vehicle; the vehicle bus is used for acquiring operation input data of a master driver on a to-be-deployed vehicle and vehicle operation data of the to-be-deployed vehicle; the visual sensor is used for acquiring a real road condition video stream of the to-be-deployed vehicle; the simulation engine is used for displaying the real road condition video stream on the simulation display device; and the simulation engine is used for generating a simulation driving result according to the vehicle operation data, the operation input data and the operation behavior data based on the data processing rule. The effect of improving the convenience and authenticity of simulation driving is achieved, and the simulation driving experience of a user is improved while the capacity of the user for processing complex traffic conditions is improved.

Description

Technical Field

[0001] This invention relates to the field of driving simulation technology, and in particular to a driving simulation system, method, device, medium and product based on real vehicle conditions. Background Technology

[0002] Driving skills are closely related to road safety. To ensure driving safety, users usually need to undergo driving training before actually driving.

[0003] In existing technologies, driving training for users is usually conducted by structurally modifying vehicles to integrate various specialized sensors, loading virtual driving scenarios, and constructing a closed and independent training vehicle system.

[0004] However, this method results in each vehicle only serving a specific model or teaching scenario, leading to high deployment costs. In addition, the driving scenarios provided rely on computer-generated virtuality, resulting in poor realism in simulated driving and affecting the user's driving experience. Summary of the Invention

[0005] This invention provides a simulation driving system, method, device, medium, and product based on real vehicle conditions, so as to achieve plug-and-play functionality of the simulation driving system without making any modifications to the original structure of the vehicle, thereby improving the convenience and realism of simulation driving, enhancing the ability of simulation driving users to handle complex traffic situations, and improving the user simulation driving experience.

[0006] According to one aspect of the present invention, a simulated driving system is provided, the system comprising: a vehicle bus, a vehicle diagnostic interface, a vision sensor, a simulation engine, a simulation display device, and a simulation operation device; wherein,

[0007] The vehicle diagnostic interface and the vehicle bus are both connected to the vehicle to be deployed, and the vision sensor is arranged on the vehicle to be deployed.

[0008] The vehicle diagnostic interface is used to determine data processing rules that are compatible with the vehicle to be deployed, and to send the data processing rules to the simulation engine.

[0009] The vehicle bus is used to acquire the operation input data of the main driver on the vehicle to be deployed and the vehicle operation data of the vehicle to be deployed, and to send the operation input data and the vehicle operation data to the simulation engine.

[0010] The visual sensor is used to acquire a real road condition video stream of the vehicle to be deployed and send the real road condition video stream to the simulation engine.

[0011] The simulation engine is used to display the real road condition video stream on the simulation display device and to receive data on the operation behavior of the simulated driving user on the simulation operation device.

[0012] The simulation engine is used to generate simulated driving results for the simulated driving user based on the data processing rules, the vehicle operation data, the operation input data, and the operation behavior data.

[0013] According to another aspect of the present invention, a simulated driving method is provided, the method comprising:

[0014] The data processing rules adapted to the vehicle to be deployed are determined through the vehicle diagnostic interface, and the data processing rules are sent to the simulation engine.

[0015] The system acquires the main driver's input data and the vehicle's operating data from the vehicle to be deployed via the vehicle bus, and sends the input data and operating data to the simulation engine. Both the vehicle diagnostic interface and the vehicle bus are connected to the vehicle to be deployed.

[0016] The visual sensor acquires a real-world road condition video stream of the vehicle to be deployed and sends the real-world road condition video stream to the simulation engine; the visual sensor is mounted on the vehicle to be deployed.

[0017] The simulation engine displays the real road condition video stream on the simulation display device and receives data on the simulated driving user's operation behavior on the simulation operation device.

[0018] Based on the data processing rules, the simulation engine generates simulation driving results for the simulated driving user according to the vehicle operation data, the operation input data, and the operation behavior data.

[0019] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0020] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the simulated driving method according to any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the simulated driving method according to any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the simulated driving method as described in any embodiment of the present invention.

[0024] The technical solution of this invention addresses the problems of high deployment costs and poor realism in existing technologies that involve structural modifications to vehicles and loading virtual driving scenarios. It obtains the main driver's input data and vehicle operation data from the vehicle bus and sends these data to the simulation engine. Both the vehicle diagnostic interface and the vehicle bus are connected to the vehicle. A vision sensor acquires a real-world road condition video stream of the vehicle and sends it to the simulation engine. The vision sensor is mounted on the vehicle. The simulation engine displays the real-world road condition video stream on a simulation display device and receives data on the simulated driving user's actions. Based on the data processing rules, the simulation engine generates a simulated driving result for the simulated driving user, taking into account the vehicle operation data, input data, and action data. This solution solves the problems of high deployment costs and poor realism in existing technologies that involve structural modifications to vehicles and loading virtual driving scenarios. It achieves this by connecting the vehicle diagnostic interface, vehicle bus, and vision sensor. This system can be deployed collaboratively on any vehicle without requiring structural modifications. It acquires real-time vehicle operation data, driver input data, and high-fidelity video streams of real-world road conditions in a realistic driving environment. Combined with user behavior data collected by the simulation device, it achieves deep integration of real-world scenarios and simulation interaction within the simulation engine. This generates simulated driving results tailored to the simulated driver, enabling analysis of their driving skills and providing driving operation suggestions to improve their capabilities. This approach avoids the high deployment costs and low versatility of traditional dedicated vehicle simulators and overcomes the distortion caused by purely virtual driving simulation devices that deviate from real traffic dynamics. It allows for the rapid conversion of ordinary vehicles into a highly realistic and adaptable driving simulation platform in a plug-and-play manner, enhancing the realism, effectiveness, and operability of driving training and evaluation. This improves the convenience and realism of simulated driving, enhances the user's ability to handle complex traffic situations, and improves the overall simulated driving experience.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the structure of a simulated driving system according to an embodiment of the present invention;

[0028] Figure 2 This is a top view of the simulated driving system provided in the embodiment of the present invention deployed in the vehicle to be deployed;

[0029] Figure 3 This is a side view of the simulated driving system provided in the embodiment of the present invention deployed in the vehicle to be deployed;

[0030] Figure 4 This is a flowchart of a simulated driving method provided according to an embodiment of the present invention;

[0031] Figure 5 This is a flowchart of a simulated driving method provided according to an embodiment of the present invention;

[0032] Figure 6 This is a flowchart of a simulated driving method provided according to an embodiment of the present invention;

[0033] Figure 7 This is a schematic diagram of the structure of an electronic device that implements the simulated driving method of this invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solution disclosed herein all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to maintain user personal information security and network security. It should also be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solution disclosed herein are all conducted with the user's knowledge and consent, and comply with relevant privacy protection regulations.

[0037] Figure 1 This is a schematic diagram of a simulated driving system provided according to an embodiment of the present invention. This embodiment is applicable to situations where users need to perform simulated driving. (Refer to...) Figure 1 The simulated driving system provided in this embodiment includes: a vehicle diagnostic interface 1, a vehicle bus 2, a vision sensor 3, a simulation engine 4, a simulation display device 5, and a simulation operation device 6. The structural composition of the simulated driving system in this embodiment will be described in detail below.

[0038] Both the vehicle diagnostic interface 1 and the vehicle bus 2 are connected to the vehicle to be deployed, and the vision sensor 3 is located on the vehicle to be deployed.

[0039] Vehicle diagnostic interface 1 is used to determine the data processing rules that are compatible with the vehicle to be deployed and send the data processing rules to simulation engine 4;

[0040] Vehicle bus 2 is used to acquire the operation input data of the main driver on the vehicle to be deployed and the vehicle operation data of the vehicle to be deployed, and send the operation input data and vehicle operation data to the simulation engine 4.

[0041] Visual sensor 3 is used to acquire real road condition video streams of the vehicle to be deployed and send the real road condition video streams to simulation engine 4.

[0042] Simulation engine 4 is used to display real road condition video streams on simulation display device 5 and to receive data on the operation behavior of simulated driving users on simulation operation device 6.

[0043] Simulation Engine 4 is used to generate simulation driving results for simulated driving users based on data processing rules, vehicle operation data, operation input data, and operation behavior data.

[0044] Among them, Vehicle Diagnostic Interface 1 refers to a standard interface (such as the OBD-II interface) that can communicate with the vehicle's Electronic Control Unit (ECU). It connects to the ECU to read vehicle status information, fault codes, and configuration parameters. Vehicle Bus 2 connects to the vehicle and is used to acquire data transmitted between various electronic modules within the vehicle, such as the driver's input data (e.g., accelerator, brake, and steering signals) and vehicle operating data (e.g., vehicle speed, engine speed). For example, Vehicle Bus 2 can be a CAN (Controller Area Network) bus. Vision Sensor 3 refers to a camera or image acquisition device that can be installed on the vehicle to capture real-time video footage of the vehicle's external environment, i.e., a real-world road condition video stream. Simulation Engine 4 refers to a software module running on a computing platform that executes simulation logic, generates visualizations, and evaluates the user's simulated driving behavior. Simulation Display Device 5 refers to a display device, such as a screen or head-mounted display, used to present the simulated driving scene, displaying the real-world road condition video stream virtually to the simulated driving user. Simulation Operation Device 6 refers to a physical interactive device used to simulate real vehicle driving controls, allowing the simulated driving user to perform driving operations. For example, the simulation control device 6 includes a steering wheel, pedals, and gear shift buttons. The operational behavior data of the simulated driving user is collected and fed back to the simulation engine 4. The simulation driving result refers to the evaluation result used to assess the driving operation of the simulated driving user.

[0045] The vehicle to be deployed refers to any type of physical vehicle that can be connected to the simulation driving system, regardless of its brand, model, power type (such as fuel, electric, or hybrid), configuration, or production year. The main driver refers to the user actually driving the vehicle to be deployed. The simulation driving user refers to the user using the simulation driving system to perform simulated driving. In this embodiment, when the simulation driving system connects to the vehicle to be deployed, it can connect to the vehicle to be deployed through the vehicle diagnostic interface 1 of the simulation driving system, and the vehicle bus 2 is connected to the vehicle to be deployed; the vision sensor 3 is arranged on the vehicle to be deployed, and the simulation operation device 6 is also arranged on the vehicle to be deployed. The simulation driving user controls the simulation operation device 6 to perform simulated driving operations, thereby realizing the "plug-and-play" vehicle simulation driving function.

[0046] In one specific implementation, the vehicle diagnostic interface 1 can read the VIN code or ECU model of the vehicle to be deployed and match it with data processing rules pre-stored in the local database. The vehicle bus 2 detects CAN messages in real time or periodically, extracts the main driver's operation commands as operation input data, and extracts the dynamic operating parameters of the vehicle to be deployed as vehicle operation data. The vision sensor 3 acquires video of the road ahead of the vehicle to be deployed through a high frame rate camera, and transmits it to the simulation engine 4 in real time after compression by a video encoder. The simulation engine 4 decodes the video stream and projects it onto the simulation display device 5 to form an immersive driving view. At the same time, it simultaneously receives the operation behavior data of the simulated driving user on the simulation operation device 6, and performs time-series alignment and behavior mapping of the main driver's operation input data and the simulated driving user's operation behavior data according to the matched data processing rules. Combined with the vehicle operation data, it outputs the simulation driving result.

[0047] To reflect the real road scene, the simulation display device 5 can be constructed using a three-fold screen and fixed in the vehicle to be deployed using elastic bands or other fixing methods, so that the simulated driving user can watch the real road condition video stream.

[0048] In another specific implementation, the vehicle diagnostic interface 1 not only identifies the vehicle model but also dynamically downloads the latest data processing rules by communicating with the cloud rule base, improving adaptability. The data acquisition of the vehicle bus 2 can employ a timestamp synchronization mechanism to ensure that operational input data and vehicle operation data are aligned with millisecond-level precision. The vision sensor 3 is arranged with multiple perspectives (such as front-view, side-view, and rear-view) and merges multiple video streams into a panoramic view, which is then fed into the simulation engine 4. The simulation engine 4 uses this panoramic video to construct an augmented reality (AR) overlay, presenting richer environmental semantic information on the simulation display device 5. The simulation engine 4 combines rule models and machine learning algorithms to compare and analyze vehicle operation data, operational input data, and operational behavior data, generating personalized driving suggestions for the simulated driving user as the simulation driving result.

[0049] The presentation of simulation driving results may include, but is not limited to: trajectory consistency score (used to measure the degree of similarity between the vehicle trajectory generated by the simulation driving user and the actual driving trajectory of the main driver in terms of path, speed, acceleration, etc.); operation deviation index (such as the amount of deviation from standard or safe driving behavior, such as excessive pedal pressing, oversteering or understeering, improper gear shifting timing, etc.); risk behavior identification (to determine whether there are potential dangerous operations such as rapid acceleration, emergency braking, lane departure, and following too closely); driving style matching degree (analyzing whether the simulation driving user's control rhythm, response delay, force control, and other parameters are close to those of the main driver or ideal driving model); task completion quality (such as in lane changing, parking, and obstacle avoidance, assessing whether the user completes the driving task safely and efficiently as required); feedback suggestions or driving tips (used to help simulation driving users improve their driving habits) and one or more combinations thereof, so as to comprehensively and objectively reflect the simulation driving user's operation ability and safety awareness through simulation driving results.

[0050] The technical solution of this invention involves determining data processing rules adapted to the vehicle to be deployed through vehicle diagnostic interface 1 and sending these rules to simulation engine 4; acquiring the main driver's operation input data and vehicle operation data of the vehicle to be deployed through vehicle bus 2 and sending these data to simulation engine 4; both vehicle diagnostic interface 1 and vehicle bus 2 are connected to the vehicle to be deployed; acquiring a real road condition video stream of the vehicle to be deployed through vision sensor 3 and sending this video stream to simulation engine 4; vision sensor 3 is installed on the vehicle to be deployed; simulation engine 4 displays the real road condition video stream on simulation display device 5 and receives operation behavior data of the simulated driving user on simulation operation device 6; and simulation engine 4 generates simulated driving results for the simulated driving user based on data processing rules, vehicle operation data, operation input data, and operation behavior data. This solves the problems of high deployment costs and poor simulation driving realism in existing technologies that involve structural modifications to vehicles and loading virtual driving scenarios. It achieves this by using vehicle diagnostic interface 1 and vehicle bus 2... Deployed in conjunction with visual sensor 3 on any vehicle to be deployed, it acquires real-time vehicle operation data, driver input data, and high-fidelity video streams of real road conditions in a realistic driving environment without requiring structural modifications to the vehicle. Combined with the user behavior data collected by the simulation operation device 6, it achieves deep integration of real-world scenarios and simulation interaction in the simulation engine 4, generating simulation driving results for the user. This allows for analysis of the user's driving skills based on the simulation results, providing driving operation suggestions and improving the user's driving ability. This not only avoids the high deployment costs and low versatility of traditional dedicated vehicle simulators but also overcomes the simulation distortion caused by pure virtual driving simulation devices that are detached from real traffic dynamics. It enables the rapid transformation of ordinary vehicles into a highly realistic and adaptable driving simulation platform in a plug-and-play manner, improving the authenticity, effectiveness, and operability of driving training or assessment, enhancing the convenience and realism of simulated driving, improving the user's ability to handle complex traffic situations, and enhancing the user's simulated driving experience.

[0051] Based on the above technical solution, the simulation operation device 6 includes at least: a pedal simulation device, a steering simulation device, and a gear simulation device;

[0052] Simulation engine 4 is used to receive data on the operational behavior of the simulated driving user on multiple simulation operation devices 6 during the simulated driving process.

[0053] Among them, the pedal simulation device refers to a device used to simulate the accelerator, brake, or clutch pedals in a real vehicle. It can sense and output information such as the user's pedal position, travel distance, speed, and state, and can also provide force feedback to enhance the realism of operation. The steering simulation device refers to a device used to simulate a real steering wheel, used to collect the angle, angular velocity, and applied torque of the simulated driver's steering wheel rotation. The gear shift simulation device refers to a device used to simulate a vehicle's gear lever or shifting mechanism, used to identify the user's currently selected gear, the timing and sequence of shifting actions, and the shift point, etc. Operation behavior data refers to the operation signals executed by the simulated driver in the simulation environment through the simulation operation device 6 (such as the pedal simulation device, steering simulation device, and gear shift simulation device), specifically including but not limited to one or more of the following: pedal position information, pedal travel information, pedal change speed, pedal state, force feedback data, steering wheel angle, steering angular velocity, steering torque, current gear state, shift duration, shift sequence, and gear shift point.

[0054] In one specific implementation, the pedal simulation device uses a high-precision linear potentiometer or magnetostrictive sensor to collect pedal position and travel in real time, and combines this with a microcontroller to calculate the pedal change speed and state. The steering simulation device is equipped with a rotary encoder and torque sensor to synchronously acquire the steering wheel angle, angular velocity, and steering torque applied by the user, and provides force feedback matching the load of the simulated vehicle through a motor drive. The gear simulation device uses a mechanical linkage structure in conjunction with a position switch or Hall sensor to accurately capture the gear status, shift sequence, and timing. All operational behavior data are transmitted in real time to the simulation engine 4 via a high-speed communication interface (such as USB or CAN). The simulation engine 4, based on a preset vehicle dynamics model and driving scenario, performs fusion analysis on these multi-dimensional operational behaviors to generate simulated driving results that are highly consistent with real driving.

[0055] In another specific implementation, each simulation operation device 6 integrates intelligent sensing and edge computing modules, which can locally filter, calibrate, and time-track the raw signals before sending the structured operation behavior data to the simulation engine 4 via wireless communication (such as Wi-Fi 6 or 5G), improving system deployment flexibility. The pedal simulation device supports adaptive damping adjustment, dynamically adjusting the force feedback intensity according to vehicle type or road conditions. The steering simulation device introduces a six-degree-of-freedom motion platform linkage mechanism, so that the steering wheel feedback not only reflects the steering system resistance but also incorporates the inertial effect brought about by changes in vehicle posture. The gear shift simulation device uses a fully electronic shift simulator, supports switching between automatic and manual modes, and can record subtle shift hesitations or misoperations. After receiving this refined operation behavior data, the simulation engine 4 combines artificial intelligence models to conduct an in-depth assessment of the user's driving style, operational proficiency, and potential risks, outputting more targeted simulated driving results.

[0056] For example, see Figure 2 The simulation display device 5 can be positioned behind the front seat 1 of the vehicle to be deployed. The steering simulation device is placed behind the simulation display device 5. The gear shift simulation device can be placed horizontally on the right side of the simulated driver's body for easy operation during simulated driving. The pedal simulation device can be placed in front of the rear passenger's foot area during actual installation. It should be noted that... Figure 2 The deployment location of the simulated driving system shown is only an example; the specific deployment location can be set according to the driving operation needs of the simulated driving user. A side view of the simulated driving system in the vehicle to be deployed can be found in [reference needed]. Figure 3 .

[0057] The technical solution provided in this embodiment constructs a high-fidelity simulation operation device 6 that includes three major control elements: pedal, steering, and gear shift. It also comprehensively collects multi-dimensional operation behavior data, including position, speed, force, and timing, thereby achieving precise acquisition of the simulated driving behavior of the simulated driving user. This not only enhances the realism and immersion of the simulated driving simulation but also ensures the accuracy of subsequent driving behavior evaluation.

[0058] Based on the above technical solution, vehicle diagnostic interface 1 is specifically used to obtain the vehicle identifier of the vehicle to be deployed and determine the data processing rules based on the vehicle identifier.

[0059] Among them, vehicle identification is used to uniquely identify the vehicle. Vehicle identification includes, but is not limited to, one or more combinations of vehicle identification number (VIN), ECU model, model year, engine type, drive type, etc., used to distinguish vehicles of different brands, models, or configurations. Data processing rules refer to a set of pre-set data parsing, mapping, transformation, or fusion logic for a specific vehicle type, used to guide the simulation engine 4 on how to correctly understand and use the operational input data, operating data, and sensor data from that vehicle.

[0060] In one specific implementation, the vehicle diagnostic interface 1 can proactively request the ECU of the vehicle to be deployed via a standard protocol (such as ISO 15765 or SAE J1979) to read its vehicle identifier. This vehicle identifier is then matched against a locally stored vehicle database to retrieve the corresponding data processing rules, such as CAN message parsing templates, pedal signal mapping coefficients, and steering system response curves. Upon successful matching, the vehicle diagnostic interface 1 packages the complete data processing rules and sends them to the simulation engine 4, ensuring that subsequently acquired vehicle bus 2 data and operational inputs can be accurately interpreted and applied.

[0061] In another specific implementation, vehicle diagnostic interface 1 can also upload vehicle identification to a cloud-based rule management platform. The cloud platform dynamically generates or optimizes data processing rules applicable to the specific vehicle configuration based on a big data model, and then sends these rules back to the local simulation driving system. This setup ensures differentiated adaptation for different configurations of the same vehicle model (such as the sports version and the comfort version), improving rule accuracy. After receiving the data processing rules, vehicle diagnostic interface 1 can securely transmit them to simulation engine 4, enabling the simulation process to accurately reproduce the unique handling characteristics and response logic of the vehicle to be deployed.

[0062] The technical solution of this invention automatically obtains the vehicle identifier of the vehicle to be deployed through the vehicle diagnostic interface 1 and determines the data processing rules accordingly, realizing an intelligent adaptation mechanism based on the real vehicle identity. This not only reduces the risk of manual configuration errors, but also improves the accuracy and consistency of the simulation engine 4 in modeling vehicle behavior, ensuring that results that are closer to the performance of real vehicles can be obtained, thereby ensuring the accuracy of the evaluation of the simulated driving behavior of simulated driving users.

[0063] Based on the above technical solutions, simulation engine 4 includes:

[0064] The data parsing unit is used to parse and obtain vehicle operation data, operation input data, and operation behavior data at the same time based on data processing rules.

[0065] The trajectory analysis unit is used to fit the first vehicle trajectory corresponding to the main driver and the second vehicle trajectory corresponding to the simulated driving user based on the vehicle operation data, operation input data and operation behavior data at the same time.

[0066] The simulation driving result determination unit is used to determine the simulation driving result based on the driving trajectory of the first vehicle and the driving trajectory of the second vehicle.

[0067] The data parsing unit, a functional module within the simulation engine 4, decodes, aligns, and structures raw data from the vehicle bus 2, diagnostic interface, and simulation operation device 6 according to pre-defined data processing rules, extracting vehicle operation data, operation input data, and operation behavior data at the same timestamp. Vehicle operation data refers to the state information generated by the vehicle during actual driving, such as speed, acceleration, yaw rate, and position coordinates, reflecting the vehicle's true dynamic data. Operation input data refers to the control commands applied to the vehicle by the driver through pedals, steering wheel, gear lever, etc., during actual driving, collected via the vehicle bus 2. The trajectory analysis unit, another functional module in the simulation engine 4, reconstructs the actual driving trajectory of the vehicle under the driver's control (first vehicle driving trajectory) and the expected driving trajectory of the virtual vehicle under the simulated driving user's control (second vehicle driving trajectory) based on the three types of data parsed at the same time. The first vehicle driving trajectory is the spatiotemporal trajectory representing the actual vehicle's path on a real road, fitted based on the real vehicle operation data and the driver's operation input data. The second vehicle trajectory refers to the virtual trajectory derived from the vehicle dynamics model based on the user's operational behavior data during the simulation, under the same initial conditions and environmental constraints. The simulation driving result determination unit is a module in simulation engine 4 used to compare the first and second vehicle trajectories, calculate their deviation, consistency, or risk indicators, and generate the final simulation driving result accordingly.

[0068] In this embodiment, the data parsing unit first performs protocol parsing and time synchronization on the operation input data, vehicle operation data and operation behavior data from the vehicle bus 2 and the simulation operation device 6 according to the data processing rules obtained from the vehicle diagnostic interface 1, so as to ensure that the vehicle operation data, operation input data and operation behavior data are strictly aligned to the same time base.

[0069] Furthermore, the trajectory analysis unit can train a deep trajectory prediction network using historical real trajectory data. Vehicle operation data and operation input data are input into the trajectory prediction network to output a first vehicle trajectory. Operation behavior data is input into the trajectory prediction network to output a second vehicle trajectory. Alternatively, by combining a real-time vehicle dynamics simulator to process operation behavior data, a second vehicle trajectory that more closely conforms to physical laws can be generated.

[0070] Furthermore, the data parsing unit can not only perform basic time alignment and field mapping, but also introduce semantic enhancement mechanisms. For example, it can identify high-level driving events such as rapid acceleration, emergency braking, or lane-changing intentions, and use these identified driving events as prior constraints for trajectory fitting. At this point, prior constraints, vehicle operation data, and operation input data can be input into the trajectory prediction network to output the first vehicle trajectory. Operation behavior data and prior constraints can be input into the trajectory prediction network to output the second vehicle trajectory. Alternatively, by combining a real-time vehicle dynamics simulator to process operation behavior data and prior constraints, a second vehicle trajectory that better conforms to physical laws can be generated.

[0071] The simulation driving result determination unit calculates the differences between the first and second vehicle trajectories in terms of lateral deviation, longitudinal speed matching, and curvature consistency, outputting structured simulation driving results. This unit also incorporates a multi-scale evaluation strategy, focusing not only on instantaneous trajectory deviations but also analyzing the consistency of driving strategies over long periods. Furthermore, it considers the safety domain boundaries to determine the existence of potential collision or loss-of-control risks, thereby generating more interpretable and instructive simulation driving results.

[0072] The technical solution provided in this embodiment, by constructing a collaborative architecture of a data parsing unit, a trajectory analysis unit, and a simulation driving result determination unit in the simulation engine 4, achieves the fitting of a first vehicle driving trajectory corresponding to the main driver and a second vehicle driving trajectory corresponding to the simulated driving user using vehicle operation data, operation input data, and operation behavior data at the same time. This not only ensures the precise alignment of real driving and simulation operation in the spatiotemporal dimension, but also transforms subjective operation into objective and quantifiable evaluation indicators through a dual trajectory comparison mechanism, ensuring the accuracy of the simulation driving evaluation of the simulated driving user.

[0073] Based on the above technical solution, the trajectory analysis unit is specifically used to fit the first vehicle driving trajectory corresponding to the main driver by using the vehicle dynamics model and based on the vehicle operation data and operation input data at the same time, and to fit the second vehicle driving trajectory corresponding to the simulated driving user based on the operation behavior data at the same time.

[0074] Among them, the vehicle dynamics model refers to the mathematical or physical model used to describe how a vehicle generates motion response under the action of control inputs (such as steering, throttle, braking), which may include longitudinal, lateral and yaw motion equations, and is used to predict or infer the vehicle's driving trajectory.

[0075] In this embodiment, the vehicle kinematics model is used to fuse the main driver's operation input data with the real vehicle operation data to deduce a high-precision first vehicle trajectory. At the same time, the operation behavior data of the simulated driving user is input into the same vehicle kinematics model to generate a second vehicle trajectory under the same initial state and real road condition video stream constraints.

[0076] In this embodiment, the trajectory analysis unit can use vehicle operation data at the same moment as initial state input, combined with corresponding operation input data (such as steering wheel angle and throttle opening), and perform forward integration using the vehicle dynamics model to gradually reconstruct the continuous motion path of the vehicle in the real environment under the control of the main driver, forming the first vehicle driving trajectory. Simultaneously, the trajectory analysis unit can use the simulated driver's operation behavior data at the same moment (such as pedal change speed, steering torque, and shift sequence) as control input, feeding it into the same vehicle dynamics model to deduce its possible driving path in the simulation environment, generating the second vehicle driving trajectory. Both trajectories are output in the form of high-frequency sampling points to ensure sufficient temporal resolution for subsequent comparisons.

[0077] It should be noted that when the trajectory analysis unit generates the first vehicle's trajectory using the vehicle dynamics model, it not only considers complex factors such as tire force, suspension characteristics, and load transfer, but also integrates real road condition information (such as curve curvature, slope, and friction coefficient estimation) obtained from the visual sensor 3 as environmental constraints. For the first vehicle's trajectory, the trajectory analysis unit can use a state estimation algorithm (such as extended Kalman filtering) to fuse vehicle operation data with operational input data based on environmental constraints, thereby reproducing the most physically accurate trajectory.

[0078] The technical solution provided in this embodiment uses a trajectory analysis unit to fit the driving trajectories corresponding to the main driver and the simulated driving user based on a unified vehicle dynamics model, thereby realizing a dual-track parallel trajectory reconstruction mechanism. This not only ensures the consistency of the evaluation benchmark, but also enables the simulation driving results to accurately reflect the differences between the simulated driving user and the main driver's control and actual vehicle driving in terms of path tracking, control coordination, and risk prediction, thereby enhancing the objectivity, interpretability, and accuracy of the simulation driving evaluation.

[0079] Based on the above technical solutions, simulation engine 4 runs in the cloud.

[0080] The cloud refers to a pool of computing, storage, and network resources provided by a remote data center, which provides an elastic and scalable service environment to the simulation engine 4 on demand via the Internet, supporting high-concurrency, high-performance, or large-scale data processing tasks.

[0081] In this embodiment, the simulation engine 4 can be deployed on a virtual server cluster of a cloud platform. All vehicle operation data and operation input data from the vehicle to be deployed, as well as operation behavior data from the local simulation operation device 6, are uploaded to the cloud in real time through a secure encrypted channel. The cloud-based simulation engine 4 utilizes its powerful parallel computing capabilities to perform the entire process of data parsing, trajectory fitting, and result evaluation, and sends the generated simulated driving results and real road condition video streams back to the simulation display device 5. The advantage of running the simulation engine 4 in the cloud is that it enables scenarios where multiple locations and multiple simulated driving users can simultaneously access the simulated driving environment.

[0082] In addition, the cloud can leverage its massive storage capacity to continuously accumulate trajectory data from different vehicle models and drivers, thereby constantly optimizing vehicle dynamics models and determining the evaluation algorithms used to determine the simulation driving results.

[0083] The technical solution provided in this embodiment, by running simulation engine 4 in the cloud, not only overcomes the limitations of local hardware resources and enables large-scale deployment of high-precision, high-complexity simulation tasks, but also improves the consistency, maintainability, and safety of the simulated driving system through centralized management. Simultaneously, relying on the elastic scaling and coverage capabilities of the cloud platform, it supports concurrent simulated driving across regions, multiple terminals, and multiple vehicles, ensuring the efficiency and response speed of data processing during simulated driving.

[0084] Based on the above technical solution, the pedal simulation device is a first box-type operation module constructed using lightweight materials;

[0085] A pedal simulation device is used to simulate pedal operation during simulated vehicle driving.

[0086] The pedal simulation device refers to a physical interactive device used to simulate the operation and function of real vehicle pedals (such as accelerator, brake, and clutch) in a simulated driving environment, allowing simulated driving users to perform pedal operations. Lightweight materials refer to engineering materials with low density and light weight but sufficient structural strength and durability, such as high-strength aluminum alloys, carbon fiber composites, or engineering plastics. The first box-type operation module refers to the pedal simulation device adopting an integrated closed box structure (i.e., "box-type") design, which integrates sensors, transmission mechanisms, and feedback actuators internally.

[0087] In this embodiment, the pedal simulation device can be constructed as a compact, integrated first box-type operating module using lightweight materials (such as an aluminum alloy frame combined with a carbon fiber panel). High-precision displacement sensors and strain gauges are embedded inside to collect pedal position, stroke, and pedaling force in real time.

[0088] This pedal simulation device can simulate the mechanical damping characteristics of a real pedal through a linkage or linear guide mechanism, and can also be equipped with a small electromagnetic brake to provide dynamic force feedback. The entire pedal simulation device integrates sensors to accurately reproduce the operation feel and response logic of the accelerator or brake pedal during simulated driving. The sensors transmit the data of the simulated driving user's operation behavior of the pedal simulation device to the simulation engine 4.

[0089] The first box-type operation module integrates intelligent sensing and edge processing units within a lightweight housing. It not only collects pedal position information, travel change speed, and state switching signals, but also performs signal filtering, nonlinear correction, and timing marking locally. The structured operation behavior data is then wirelessly transmitted to the simulation engine 4.

[0090] The pedal simulation device supports a modular quick-release design and can be flexibly configured as a single pedal (such as electric vehicle mode), a dual pedal (accelerator + brake), or a triple pedal (including clutch) layout. By changing the internal springs or adjusting the feedback motor parameters, it can be adapted to the pedal force curves of different vehicle models. The use of lightweight materials ensures structural rigidity while reducing overall inertia, thereby improving the sensitivity and realism of user operation.

[0091] The technical solution provided in this embodiment improves the portability, durability, and configuration flexibility of the driving simulation system by designing the pedal simulation device as a first box-type operating module constructed using lightweight materials. This design allows the driving simulation system to be deployed more efficiently to any vehicle to be deployed, while the high-fidelity pedal feedback mechanism enhances user immersion and operational accuracy, thereby improving the effectiveness, adaptability, and user experience of the entire driving simulation system.

[0092] Based on the above technical solution, the steering simulation device is a simulated steering wheel constructed using lightweight materials; the steering simulation device is horizontally fixed by elastic bands or metal clips; and the operating height of the steering simulation device is adjusted using a telescopic structure.

[0093] Steering simulation device, used to simulate steering wheel operation during vehicle driving simulation.

[0094] The steering simulation device refers to an interactive device used to simulate the operation of a real vehicle's steering wheel in a simulated driving environment. It can collect the steering wheel angle, steering speed, and torque applied by the user, and can also provide force feedback to recreate the realistic steering feel. The simulated steering wheel is the main body of the steering simulation device; its shape, size, and grip are as close as possible to a real vehicle's steering wheel, and it serves to support sensors and feedback mechanisms. The elastic band is a flexible fastener with elastic tensile properties, used to provide cushioning constraints, facilitating quick installation of the steering simulation device and allowing for minor adjustments. The metal clamp is a rigid clamping mechanism made of metal with an adjustable fastening structure, used to securely fix the steering simulation device to a bracket or other support surface. The telescopic structure refers to a mechanical component (such as a sleeve-type column, gas spring, or threaded lifting rod) whose length can be adjusted vertically, used to change the height position of the steering simulation device relative to the user's seat to accommodate different heights or seating postures.

[0095] In this embodiment, the steering simulation device is integrally molded from lightweight materials to simulate a steering wheel. It integrates a high-precision rotary encoder and torque sensor to capture the steering wheel angle, angular velocity, and steering force applied by the user in real time. The steering simulation device is securely and horizontally fixed to the dashboard or bracket via a metal clamp connected to its bottom, ensuring no displacement during aggressive steering maneuvers. Simultaneously, its support column employs a multi-section telescopic structure, allowing the user to manually adjust the steering wheel's operating height using a tightening knob to suit different body types or seat positions.

[0096] The steering simulation device can also be secured using a high-elasticity elastic band with adjustable anchor points, flexibly binding it to the support frame. This flexible fixing method not only simplifies the installation process but also absorbs some impact energy in the event of an accidental collision, improving safety. Furthermore, the height of the steering simulation device can be adjusted via a telescopic structure with built-in gas springs or electric push rods, supporting stepless smooth lifting and lowering. It can even be linked to user identification, automatically memorizing and resetting personalized height settings.

[0097] For example, the steering simulation device is used to simulate steering wheel-related operations during simulated car driving, providing steering and front and rear wheel angle position inputs to the driving behavior analysis terminal. It can be horizontally fixed using elastic bands, metal clips, or other fixing methods, and the height of the steering simulation device can be adjusted using a telescopic structure to place the steering wheel operating blocks in an easily operable position (such as above the simulated driver's lap, behind the front seat, etc.).

[0098] The technical solution provided in this embodiment constructs a steering simulation device as a simulated steering wheel made of lightweight materials, and fixes it horizontally with elastic bands or metal clips. It is further enhanced by an adjustable telescopic structure to achieve high adaptability. This ensures operational stability while improving the deployment flexibility, human-machine compatibility, and safety of the steering simulation device. It meets the personalized needs of different user groups, enhances the realism and comfort of steering wheel operation during simulated driving, and enhances the realism and immersion of steering operation.

[0099] Based on the above technical solution, the gear simulation device is a second box-type operation module constructed using lightweight materials;

[0100] Gear shifting device is used to simulate gear shifting during simulated vehicle driving.

[0101] The gear shifting simulation device refers to a physical interactive device used to simulate the operation of a real vehicle's gear lever or shifting mechanism in a simulated driving environment. It can recognize the gear selection, shifting actions, and timing performed by the user to recreate the shifting experience of a manual or automatic transmission. The second box-type operation module refers to the gear shifting simulation device's integrated enclosed box structure design, which integrates a gear recognition mechanism, position sensors, and feedback components. Gear shifting operation refers to the process by which the simulated driving user operates the gear shifting simulation device to shift from one gear to another, including gear status (such as P / R / N / D or 1 / 2 / 3 / 4 / 5 / R), shifting sequence, shifting duration, and shifting timing.

[0102] In this embodiment, the gear shift simulation device is constructed as a second box-type operating module using lightweight materials. Its outer shell is made of high-strength engineering plastic, and it houses a mechanical linkage shift fork and a Hall sensor array to accurately capture the user's shifting path and gear placement. The gear shift simulation device can also switch between gear operation modes (automatic or manual) by changing the panel or adjusting the internal limit structure. When the simulated driving user performs gear shifting operations, the gear shift simulation device records the current gear status, shift start and end times, shift sequence, and operation duration in real time, and transmits this operational data to the simulation engine 4 via a wired interface or wirelessly.

[0103] Alternatively, the second box-type operation module can integrate an intelligent microcontroller and a wireless communication unit within a lightweight housing. This allows for local processing of raw sensor signals and semantic recognition of gear shifting actions (such as determining whether a gear shift is valid, whether there is a mis-engagement, or hesitation). The gear shift simulation device can employ a magnetic or snap-on quick-release design, using a bottom metal clip or elastic strap for rapid horizontal fixation. All gear shifting operation data is pre-processed at the edge and then uploaded to the cloud simulation engine 4 via a low-latency wireless link.

[0104] The technical solution provided in this embodiment improves the overall portability, compatibility, and realism of the human-computer interaction of the simulated driving system by designing the gear shift simulation device as a second box-type operation module constructed using lightweight materials, while ensuring the accuracy of gear shift operation recognition. It can be flexibly adapted to both automatic and manual transmission vehicles through structural or software configuration, achieving "plug-and-play" deployment capabilities. Furthermore, by combining refined data collection on gear shift timing, sequence, and status, the simulation engine 4 can comprehensively evaluate the rationality of the simulated driving user's gear shifts and driving proficiency, ensuring the accuracy of the assessment of the simulated driving user's driving behavior. This allows for timely and effective operational reminders to the simulated driving user, improving the driving experience of the simulated driving model.

[0105] As an optional embodiment of the above embodiments, specific application scenario examples are provided to enable those skilled in the art to further understand the technical solutions of the embodiments of the present invention. Specifically, please refer to the following detailed content.

[0106] The technical solution provided by the embodiments of the present invention provides a universal portable simulation driving system based on real vehicle conditions. Based on the "plug-and-play, vehicle-separated" simulation driving system, it can realize the transformation of dynamic data of any vehicle in real driving into a high-fidelity and safe driving training environment exclusive to rear-seat simulation driving users in real time.

[0107] See Figure 4 The specific implementation methods for achieving simulated driving based on a simulated driving system can be as follows:

[0108] The system connects to the vehicle to be deployed via a standardized vehicle diagnostic interface (such as OBD-II). The simulated driving system automatically performs identity recognition and protocol matching: it reads the unique identification code (VIN) of the vehicle to be deployed and automatically loads the corresponding data processing rules from the built-in vehicle model protocol library accordingly. This process requires no manual configuration, achieving "plug-and-play" and seamless integration between the simulated driving system and any vehicle model, facilitating the parsing of vehicle operation data and operational input data based on data processing rules.

[0109] A temporary in-vehicle vision sensor 3 is deployed in the vehicle to acquire a real-world road condition video stream. This video stream is then sent to a rear-seat simulation display device 5, presenting a virtual driving environment that is completely synchronized with the real world. The vehicle behavior in this virtual driving environment is directly driven by real-world road condition data, mirroring the real-time road conditions experienced by the current driver. This ensures high fidelity in the simulation and aids in desensitization training for rear-seat simulated driving users.

[0110] The rear-seat simulated driver operates the vehicle in a synchronized virtual driving environment via wirelessly connected simulated control blocks (steering wheel, pedals, etc.). The simulated driving system collects real-time vehicle operation data and driver input data (including but not limited to vehicle speed, steering wheel angle, accelerator / brake opening, and gear position) from the vehicle to be deployed via a self-adaptive vehicle bus 2. This multi-source data (including vehicle operation data and driver input data) is input into a lightweight simulation engine 4, which determines the simulated driving results for the simulated driver.

[0111] The method for determining the simulation driving results for simulated driving users is as follows: Based on the different operational inputs of the front-seat driver and the rear-seat simulated driving user at the same time, under the same initial vehicle state and road conditions, the simulated driving system predicts two future short-term trajectories (i.e., the first vehicle trajectory and the second vehicle trajectory) using a built-in vehicle dynamics model. By analyzing whether the two trajectories comply with safety regulations (such as whether they cross the lines or maintain a safe distance), the system intelligently judges the rationality and safety of the simulated driving user's operations, obtains the simulation driving results, and provides real-time feedback based on the simulation driving results.

[0112] See Figure 5 The simulation operation device 6 includes at least: a pedal simulation device, a steering simulation device, and a gear simulation device.

[0113] The pedal simulation device, steering simulation device, and gear shift simulation device jointly collect data on the user's actions on the simulation operation device 6, i.e., virtual driving information. This data can be transmitted to the simulation display device 5 for display via data cable or Bluetooth. A vision sensor 3 (such as an optical sensor) mounted on the vehicle to be deployed acquires a video stream of the real road conditions (i.e., the vehicle's actual driving conditions) and transmits this video stream to the simulation display device 5 for display.

[0114] The simulation display device 5 can send operation behavior data and real road condition video streams to the cloud simulation engine 4 via a wireless network. The vehicle bus 2 acquires the operation input data of the main driver on the vehicle to be deployed and the vehicle operation data of the vehicle to be deployed, and sends the operation input data and vehicle operation data to the simulation engine 4.

[0115] Based on data processing rules, the cloud-based simulation engine 4 generates simulation driving results for the simulated driving user according to vehicle operation data, operation input data, and operation behavior data. The simulation driving results can then be sent to the simulation display device 5 for display.

[0116] The technical solution provided in this embodiment enables the rapid deployment and zero-intervention startup of the simulated driving system on any vehicle by setting up a universal vehicle diagnostic interface and multiple lightweight simulation operation models. This solves the problems of poor versatility, insufficient realism, and conflict with real driving safety of traditional simulators. It achieves data interconnection and simulated driving for any vehicle model without modifying the original vehicle design, thereby improving the convenience and realism of simulated driving, enhancing the user's ability to handle complex traffic situations, and improving the user's simulated driving experience.

[0117] Figure 6 This is a flowchart of a simulated driving method according to an embodiment of the present invention. This embodiment is applicable to situations where users need to perform simulated driving. The method can be executed by a simulated driving system, including a vehicle bus, vehicle diagnostic interface, vision sensor, simulation engine, simulation display device, and simulation operation device. The simulated driving system can be implemented in hardware and / or software, and can be configured in a computing device or vehicle. Figure 6 As shown, the method includes:

[0118] S110. Determine the data processing rules that are compatible with the vehicle to be deployed through the vehicle diagnostic interface, and send the data processing rules to the simulation engine.

[0119] S120. Obtain the operation input data of the main driver on the vehicle to be deployed and the vehicle operation data of the vehicle to be deployed through the vehicle bus, and send the operation input data and the vehicle operation data to the simulation engine; the vehicle diagnostic interface and the vehicle bus are both connected to the vehicle to be deployed.

[0120] S130. Acquire the real road condition video stream of the vehicle to be deployed using a visual sensor, and send the real road condition video stream to the simulation engine; the visual sensor is arranged on the vehicle to be deployed.

[0121] S140. Display the real road condition video stream on the simulation display device through the simulation engine, and receive the operation behavior data of the simulated driving user on the simulation operation device.

[0122] S150. Based on the data processing rules, the simulation engine generates a simulation driving result for the simulated driving user according to the vehicle operation data, the operation input data, and the operation behavior data.

[0123] The simulation operation device includes at least: a pedal simulation device, a steering simulation device, and a gear simulation device;

[0124] The simulation engine is used to receive data on the operational behavior of the simulated driving user on multiple simulation operation devices during the simulated driving process;

[0125] The operational behavior data includes one or more of the following: pedal position information, pedal travel information, pedal change speed, pedal status, force feedback data, steering wheel angle, steering angular velocity, steering torque, current gear status, shift duration, shift sequence, and gear shift time.

[0126] The vehicle diagnostic interface is specifically used to obtain the vehicle identifier of the vehicle to be deployed and to determine data processing rules based on the vehicle identifier.

[0127] The simulation engine includes:

[0128] The data parsing unit is used to parse the vehicle operation data, the operation input data, and the operation behavior data at the same time based on the data processing rules.

[0129] The trajectory analysis unit is used to fit the first vehicle driving trajectory corresponding to the main driver and the second vehicle driving trajectory corresponding to the simulated driving user based on the vehicle operation data, the operation input data and the operation behavior data at the same time.

[0130] The simulation driving result determination unit is used to determine the simulation driving result based on the driving trajectory of the first vehicle and the driving trajectory of the second vehicle.

[0131] The trajectory analysis unit is specifically used to fit a first vehicle driving trajectory corresponding to the main driver using a vehicle dynamics model, based on the vehicle operation data and operation input data at the same time, and to fit a second vehicle driving trajectory corresponding to the simulated driving user based on the operation behavior data at the same time.

[0132] The simulation engine runs in the cloud.

[0133] The pedal simulation device is a first box-type operation module constructed using lightweight materials;

[0134] The pedal simulation device is used to simulate pedal operation during vehicle driving simulation.

[0135] The steering simulation device is a simulated steering wheel constructed using lightweight materials; the steering simulation device is horizontally fixed by elastic bands or metal clips; and the operating height of the steering simulation device is adjusted using a telescopic structure.

[0136] The steering simulation device is used to simulate steering wheel operation during vehicle driving simulation.

[0137] The gear simulation device is a second box-type operation module constructed using lightweight materials;

[0138] The gear shifting device is used to simulate gear shifting operations during simulated vehicle driving.

[0139] The technical solution provided in this embodiment determines data processing rules adapted to the vehicle to be deployed through a vehicle diagnostic interface and sends these rules to the simulation engine; it acquires the main driver's operation input data and the vehicle's operation data through the vehicle bus and sends these data to the simulation engine; both the vehicle diagnostic interface and the vehicle bus are connected to the vehicle to be deployed; it acquires a real-world road condition video stream of the vehicle to be deployed through a vision sensor and sends this video stream to the simulation engine; the vision sensor is mounted on the vehicle to be deployed; the simulation engine displays the real-world road condition video stream on a simulation display device and receives operation behavior data from the simulated driving user on the simulation operation device; based on the data processing rules, the simulation engine generates a simulated driving result for the simulated driving user according to the vehicle operation data, operation input data, and operation behavior data. This solves the problems of high deployment costs and poor realism in existing technologies that involve structural modifications to the vehicle and loading virtual driving scenarios. It achieves this by connecting the vehicle diagnostic interface, vehicle bus, and vision sensor... This system can be deployed collaboratively on any vehicle without requiring structural modifications. It acquires real-time vehicle operation data, driver input data, and high-fidelity video streams of real-world road conditions in a realistic driving environment. Combined with user behavior data collected by the simulation device, it achieves deep integration of real-world scenarios and simulation interaction within the simulation engine. This generates simulated driving results tailored to the simulated driver, enabling analysis of their driving skills and providing driving operation suggestions to improve their capabilities. This approach avoids the high deployment costs and low versatility of traditional dedicated vehicle simulators and overcomes the distortion caused by purely virtual driving simulation devices that deviate from real traffic dynamics. It allows for the rapid conversion of ordinary vehicles into a highly realistic and adaptable driving simulation platform in a plug-and-play manner, enhancing the realism, effectiveness, and operability of driving training and evaluation. This improves the convenience and realism of simulated driving, enhances the user's ability to handle complex traffic situations, and improves the overall simulated driving experience.

[0140] Figure 7This is a schematic diagram of the structure of an electronic device implementing the simulated driving method of this invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0141] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory 12 or a random access memory 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 12 or a computer program loaded from storage unit 18 into the random access memory 13. The random access memory 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, read-only memory 12, and random access memory 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0142] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0143] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as simulated driving methods.

[0144] In some embodiments, the simulated driving method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via read-only memory 12 and / or communication unit 19. When the computer program is loaded into random access memory 13 and executed by processor 11, one or more steps of the simulated driving method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the simulated driving method by any other suitable means (e.g., by means of firmware).

[0145] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0147] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0149] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0150] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0151] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from read-only memory 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0152] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the simulated driving method provided in any embodiment of this invention.

[0153] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0154] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0155] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A simulation driving system characterized by comprising: include: Vehicle bus, vehicle diagnostic interface, vision sensor, simulation engine, simulation display device, and simulation operation device; among which, The vehicle diagnostic interface and the vehicle bus are both connected to the vehicle to be deployed, and the vision sensor is arranged on the vehicle to be deployed. The vehicle diagnostic interface is used to determine data processing rules that are compatible with the vehicle to be deployed, and to send the data processing rules to the simulation engine. The vehicle bus is used to acquire the operation input data of the main driver on the vehicle to be deployed and the vehicle operation data of the vehicle to be deployed, and to send the operation input data and the vehicle operation data to the simulation engine. The visual sensor is used to acquire a real road condition video stream of the vehicle to be deployed and send the real road condition video stream to the simulation engine. The simulation engine is used to display the real road condition video stream on the simulation display device and to receive data on the operation behavior of the simulated driving user on the simulation operation device. The simulation engine is used to generate simulated driving results for the simulated driving user based on the data processing rules, the vehicle operation data, the operation input data, and the operation behavior data.

2. The system according to claim 1, characterized in that, The simulation operation device includes at least: a pedal simulation device, a steering simulation device, and a gear simulation device; The simulation engine is used to receive data on the operational behavior of the simulated driving user on multiple simulation operation devices during the simulated driving process; The operational behavior data includes one or more of the following: pedal position information, pedal travel information, pedal change speed, pedal status, force feedback data, steering wheel angle, steering angular velocity, steering torque, current gear status, shift duration, shift sequence, and gear shift time.

3. The system according to claim 1, characterized in that, The vehicle diagnostic interface is specifically used to obtain the vehicle identifier of the vehicle to be deployed and to determine data processing rules based on the vehicle identifier.

4. The system according to claim 1, characterized in that, The simulation engine includes: The data parsing unit is used to parse the vehicle operation data, the operation input data, and the operation behavior data at the same time based on the data processing rules. The trajectory analysis unit is used to fit the first vehicle driving trajectory corresponding to the main driver and the second vehicle driving trajectory corresponding to the simulated driving user based on the vehicle operation data, the operation input data and the operation behavior data at the same time. The simulation driving result determination unit is used to determine the simulation driving result based on the driving trajectory of the first vehicle and the driving trajectory of the second vehicle.

5. The system according to claim 4, characterized in that, The trajectory analysis unit is specifically used to fit a first vehicle driving trajectory corresponding to the main driver using a vehicle dynamics model, based on the vehicle operation data and the operation input data at the same time, and to fit a second vehicle driving trajectory corresponding to the simulated driving user based on the operation behavior data at the same time.

6. The system according to claim 1, characterized in that, The simulation engine runs in the cloud.

7. The system according to claim 2, characterized in that, The pedal simulation device is a first box-type operation module constructed using lightweight materials; The pedal simulation device is used to simulate pedal operation during vehicle driving simulation.

8. The system according to claim 2, characterized in that, The steering simulation device is a simulated steering wheel constructed using lightweight materials; the steering simulation device is horizontally fixed by elastic bands or metal clips; and the operating height of the steering simulation device is adjusted using a telescopic structure. The steering simulation device is used to simulate steering wheel operation during vehicle driving simulation.

9. The system according to claim 2, characterized in that, The gear simulation device is a second box-type operation module constructed using lightweight materials; The gear shifting device is used to simulate gear shifting operations during simulated vehicle driving.

10. A simulated driving method, characterized in that, include: The data processing rules adapted to the vehicle to be deployed are determined through the vehicle diagnostic interface, and the data processing rules are sent to the simulation engine. The system acquires the main driver's operation input data and the vehicle operation data of the vehicle to be deployed through the vehicle bus, and sends the operation input data and the vehicle operation data to the simulation engine. Both the vehicle diagnostic interface and the vehicle bus are connected to the vehicle to be deployed; The system acquires a video stream of the real road conditions where the vehicle to be deployed is located using a visual sensor, and sends the video stream of the real road conditions to the simulation engine. The vision sensor is mounted on the vehicle to be deployed. The simulation engine displays the real road condition video stream on the simulation display device and receives data on the simulated driving user's operation behavior on the simulation operation device. Based on the data processing rules, the simulation engine generates simulation driving results for the simulated driving user according to the vehicle operation data, the operation input data, and the operation behavior data.

Images