Heavy-duty cab-based simulation training processing method and system

Abstract

The present application is applicable to the technical field of driving training, and provides a heavy-duty cab-based simulation training processing method and system. The method comprises: displaying a simulated driving scenario on a display screen around a simulated heavy-duty cab, the display screen being a surrounding seamless screen, used for covering an entire driving field of view of the simulated heavy-duty cab; in response to at least two rounds of simulated driving control operations performed by a user on the simulated heavy-duty cab on the basis of the simulated driving scenario, acquiring raw driving training data corresponding to each round of simulated driving control operations; performing data processing on the raw driving training data, and obtaining target driving training data; and performing data analysis on the target driving training data, and obtaining a training result of the user. Said method enables the user to perform immersive simulated driving control operations on the simulated heavy-duty cab on the basis of the simulated driving scenario, thereby providing a required visual experience for a driver of a heavy-duty truck and improving the effectiveness of simulated driving training of the heavy-duty truck.

Description

A simulation training processing method and system based on a heavy-duty cockpit

[0001] This application claims priority to Chinese Patent Application No. 202510376770.4, filed on March 27, 2025, entitled "A Simulation Training Processing Method and System Based on a Heavy-Duty Cockpit", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application belongs to the field of traffic training technology, and in particular relates to a simulation training processing method and system based on a heavy-duty cockpit. Background Technology

[0003] Currently, driving simulators are increasingly being used for vehicle training. For example, multi-degree-of-freedom motion platforms and virtual displays can be used to provide drivers with a virtual driving experience to achieve simulated driving training.

[0004] However, existing simulation training systems are mainly geared towards light vehicle driver training, providing only simple virtual displays to simulate the vehicle's driving environment. They fail to provide the necessary visual experience for heavy truck drivers, severely impacting the effectiveness of heavy truck simulation driving training. Summary of the Invention

[0005] This application provides a simulation training method and system based on a heavy-duty driver's cab, which can provide heavy-duty truck drivers with the necessary visual experience and improve the effectiveness of heavy-duty truck simulation driving training.

[0006] In a first aspect, embodiments of this application provide a simulation training processing method based on a heavy-duty cockpit, including:

[0007] In response to the user's activation training operation on the simulated heavy-duty cockpit, the simulated driving scene is displayed on the screens around the simulated heavy-duty cockpit. The screens are enclosed seamless screens used to cover the entire driving field of vision of the simulated heavy-duty cockpit.

[0008] In response to at least two rounds of simulated driving control operations performed by the user on the simulated heavy-duty cockpit based on the simulated driving scenario, the system acquires the original driving training data corresponding to each round of simulated driving control operations; and processes the original driving training data corresponding to at least two rounds of simulated driving control operations to obtain the target driving training data.

[0009] Data analysis is performed on the target driving training data to obtain the user's training results. The training results are used to characterize whether there are positive changes in the user's driving control during at least two rounds of simulated driving control.

[0010] As an embodiment of the first aspect of this application, the simulated driving scenario includes at least a simulated driving environment, and the method further includes the following steps before displaying the simulated driving scenario on a display screen around a simulated heavy-duty cockpit:

[0011] Acquire raw driving environment data, wherein the raw driving environment data is obtained by collecting outdoor driving environment data in at least one of the following ways: different heights and different angles;

[0012] The original driving environment data is reconstructed in three dimensions using multi-view stereo reconstruction and point cloud generation technology to generate the original point cloud data of the original driving environment data.

[0013] The raw point cloud data is processed to obtain a simulated driving environment. The data processing includes at least one of the following: mesh generation, texture mapping, lighting adjustment, and format conversion.

[0014] As an embodiment of the first aspect of this application, the simulated driving scenario further includes: a simulated blind spot area and simulated dynamic events, wherein the simulated dynamic events include at least dynamic traffic signal events and dynamic obstacle events.

[0015] As an embodiment of the first aspect of this application, data processing is performed on the original driving training data corresponding to at least two rounds of simulated driving control operations to obtain target driving training data, including:

[0016] The original driving training data of each round of simulated driving control operation in at least two rounds of simulated driving control operation are preprocessed to obtain the preprocessed training data of each round of simulated driving control operation.

[0017] Model the preprocessed training data corresponding to at least two rounds of simulated driving control operations to obtain the initial generalized linear mixture model corresponding to at least two rounds of simulated driving control operations.

[0018] Select the target generalized linear mixture model from the initial generalized linear mixture models corresponding to at least two rounds of simulated driving control operations;

[0019] The target generalized linear mixture model was used as the target driving training data.

[0020] As an embodiment of the first aspect of this application, a target generalized linear mixture model is selected from the initial generalized linear mixture models corresponding to at least two rounds of simulated driving control operations, including:

[0021] The stepwise backward regression technique is used to select driving training data that meet the preset parameter conditions from the initial generalized linear mixture model to obtain the target generalized linear mixture model.

[0022] As an embodiment of the first aspect of this application, obtaining the original driving training data corresponding to each round of simulated driving control operation includes at least one of the following:

[0023] The original driving speed corresponding to each round of simulated driving control operation is collected by a speed sensor configured in the simulated heavy-duty cockpit.

[0024] The lane departure sensor, configured in the simulated heavy-duty cockpit, collects the raw lateral position data corresponding to each round of simulated driving control operation;

[0025] By using a steering wheel sensor configured in the simulated heavy-duty cockpit, the steering wheel angle value corresponding to each round of simulated driving control operation is collected;

[0026] The braking speed and braking time corresponding to each round of simulated driving control operation are collected by a pedal sensor configured on the brake pedal in the simulated heavy-duty cockpit.

[0027] An eye tracker configured in a simulated heavy-duty cockpit is used to collect fixation point data, blink frequency, and eye closure time corresponding to each round of simulated driving control operations.

[0028] An eye tracker is installed in the simulated heavy-duty cockpit to collect head direction data, eye movement distance, and eye movement time corresponding to each round of simulated driving control operation;

[0029] The accelerator pedal sensor, configured in the simulated heavy-duty cockpit, collects the applied accelerator pressure value corresponding to each round of simulated driving control operation.

[0030] As an embodiment of the first aspect of this application, the original driving training data of each round of simulated driving control operation in at least two rounds of simulated driving control operation are preprocessed to obtain preprocessed training data for each round of simulated driving control operation, including at least one of the following:

[0031] Calculate the maximum driving speed and speed standard deviation corresponding to each round of simulated driving control operation based on the original driving speed corresponding to each round of simulated driving control operation;

[0032] Based on the original lateral position data and lane center position data corresponding to each round of simulated driving control operation, calculate the lateral position deviation data corresponding to each round of simulated driving control operation.

[0033] Based on the steering wheel angle value corresponding to each round of simulated driving control operation, calculate the angular velocity, angular acceleration and angular direction data corresponding to each round of simulated driving control operation;

[0034] The braking force corresponding to each round of simulated driving control operation is calculated based on the braking speed corresponding to each round of simulated driving control operation, and the braking reaction time corresponding to each round of simulated driving control operation is calculated based on the braking time corresponding to each round of simulated driving control operation.

[0035] Calculate the percentage of eye closure for each round of simulated driving control operation based on the eye closure time for each round of simulated driving control operation.

[0036] The eye movement speed corresponding to each round of simulated driving control operation is calculated based on the head direction data, eye movement distance and eye movement time corresponding to each round of simulated driving control operation.

[0037] The throttle control rate for each round of simulated driving control operation is calculated based on the applied throttle pressure value corresponding to each round of simulated driving control operation.

[0038] As an embodiment of the first aspect of this application, the display screen includes a first display screen, a second display screen, and a third display screen that are seamlessly connected. The first display screen is used to display a simulated driving scene with a 180° field of view in front of the simulated heavy-duty cockpit. The second display screen is matched with the left rearview mirror of the simulated heavy-duty cockpit and is used to display a simulated driving scene with a left-side field of view of the simulated heavy-duty cockpit. The third display screen is matched with the right rearview mirror of the simulated heavy-duty cockpit and is used to display a simulated driving scene with a right-side field of view of the simulated heavy-duty cockpit.

[0039] Secondly, embodiments of this application provide a simulation training processing system based on a heavy-duty cockpit, comprising:

[0040] Simulates a heavy-duty cab, derived from the cab of a real heavy-duty truck;

[0041] The display screen is positioned around the simulated heavy-duty cockpit. It is a seamless, wraparound screen designed to cover the entire driving field of vision of the simulated heavy-duty cockpit and display the simulated driving scenario.

[0042] The data acquisition device is used to collect raw driving training data corresponding to at least two rounds of simulated driving control operations performed by the user on the simulated heavy-duty cockpit based on the simulated driving scenario.

[0043] The processor is used to process the raw driving training data corresponding to at least two rounds of simulated driving control operations to obtain target driving training data; and to analyze the target driving training data to obtain the user's training results. The training results are used to characterize whether there is a positive change in the user's driving control during at least two rounds of simulated driving control.

[0044] As an embodiment of the second aspect of this application, the simulated heavy-duty cockpit is equipped with a steering wheel, a brake pedal and an accelerator pedal, and the simulated heavy-duty cockpit is equipped with a left rearview mirror and a right rearview mirror.

[0045] As an embodiment of the second aspect of this application, the display screen includes a first display screen, a second display screen, and a third display screen that are seamlessly connected. The first display screen is used to display a simulated driving scene with a 180° field of view in front of the simulated heavy-duty cockpit. The second display screen is matched with the left rearview mirror and is used to display a simulated driving scene with a left field of view in the simulated heavy-duty cockpit. The third display screen is matched with the right rearview mirror and is used to display a simulated driving scene with a right field of view in the simulated heavy-duty cockpit.

[0046] This application provides a simulation training method and system based on a heavy-duty truck cockpit. The method includes: in response to a user's activation of training operations on the simulated heavy-duty truck cockpit, displaying a simulated driving scene on a display screen surrounding the simulated heavy-duty truck cockpit, wherein the display screen is a seamless, surround-type screen used to cover the entire driving field of view of the simulated heavy-duty truck cockpit; in response to the user performing at least two rounds of simulated driving control operations on the simulated heavy-duty truck cockpit based on the simulated driving scene, acquiring original driving training data corresponding to each round of simulated driving control operations; processing the original driving training data corresponding to the at least two rounds of simulated driving control operations to obtain target driving training data; and performing data analysis on the target driving training data to obtain the user's training results, wherein the training results are used to characterize whether there is a positive change in the user's driving control during the at least two rounds of simulated driving control. Using the above technical solution, by displaying a simulated driving scene on a display screen surrounding the simulated heavy-duty truck cockpit, and the display screen being a seamless, surround-type screen used to cover the entire driving field of view of the simulated heavy-duty truck cockpit, users can perform immersive simulated driving control operations on the simulated heavy-duty truck cockpit based on the simulated driving scene, providing heavy-duty truck drivers with the necessary visual experience and improving the effectiveness of heavy-duty truck simulated driving training. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 is a structural block diagram of a simulation training processing system based on a heavy-duty cockpit provided in an embodiment of this application;

[0049] Figure 2 is a schematic diagram of the structure of a motion platform provided in an embodiment of this application;

[0050] Figure 3 is a schematic diagram of the architecture of a simulation training processing system provided in an embodiment of this application;

[0051] Figure 4 is a flowchart illustrating a simulation training processing method based on a heavy-duty cockpit according to an embodiment of this application;

[0052] Figure 5 is a flowchart illustrating a simulation training method based on a heavy-duty cockpit, according to another embodiment of this application. Detailed Implementation

[0053] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0054] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0055] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0056] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0057] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0058] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0059] It should be noted that the information collection process (such as the facial image collection process, the training data collection process, etc.) / feature extraction process involved in this application is carried out with the user's knowledge and permission. That is, the information collection process / feature extraction process complies with the requirements of laws and regulations and does not constitute an act that harms the public interest.

[0060] It can be argued that driving simulators are increasingly being used for vehicle simulation training. For example, existing simulation training systems (such as intelligent commercial vehicle test drive platforms) can provide users with a realistic driving experience using a six-degree-of-freedom motion platform and a virtual display screen.

[0061] However, these test drive platforms primarily cater to the driving experience of light vehicles, lacking detailed evaluation of various driving performance parameters and failing to fully meet the specific requirements of heavy truck operation, such as longer braking distances, larger turning radii, and more complex operating logic.

[0062] Specifically, existing simulation training systems typically only provide simple virtual displays, such as a single or a few forward-facing displays, and cannot simultaneously display rearview and side mirror views. For example, in actual operation, heavy truck drivers need to monitor the situation in front of, to the sides, or behind the vehicle. The use of rearview mirrors is crucial, especially in complex traffic situations such as lane changes, reversing, and turning. However, existing simulation training systems, due to their limited field of view, cannot provide the panoramic view required by heavy truck drivers. This results in drivers not receiving visual feedback consistent with real driving conditions, hindering their ability to fully immerse themselves in the driving environment. External, irrelevant visual information can distract drivers and affect the accuracy of their driving performance.

[0063] Secondly, existing simulation training systems primarily simulate the driving operations of standard private cars and commercial vehicles, failing to fully reflect the physical characteristics and operational complexity of heavy-duty trucks. This results in a lack of operational feedback that matches the real-world experience of driving heavy-duty trucks, thus impacting the final training effectiveness. In real-world scenarios, heavy-duty trucks differ significantly from light-duty vehicles in braking, acceleration, steering, and manual transmission shifting. For example, heavy-duty trucks have longer braking distances, requiring earlier braking; their weight and size also lead to larger turning radii and greater operational difficulty. Furthermore, the manual shifting and acceleration responses of heavy-duty trucks are more complex, demanding higher skill from drivers.

[0064] Furthermore, existing simulation training systems typically emphasize the driver's subjective feedback on the simulation experience. Although basic operational data such as speed and steering angle are recorded, the collection and analysis of this data are often rudimentary and lack scientific evaluation and analysis algorithms. This is especially true in heavy truck driver training, where the higher complexity of operations necessitates detailed data analysis (e.g., speed standard deviation, lateral position deviation, braking force, and reaction time) to assess the driver's technical skills and ability to handle unexpected events. Therefore, existing simulation training systems struggle to provide personalized feedback or effectively improve driver skills.

[0065] Finally, existing simulation training systems typically only provide simple, static roads and basic conditions, lacking the complexity and uncertainty of actual traffic flow. They fail to dynamically simulate complex traffic conditions and cannot effectively simulate blind spots, preventing drivers from experiencing real traffic situations and thus limiting their ability to improve emergency response skills. For example, existing simulation training systems only present standard lanes and traffic signal control scenarios, failing to simulate real road events, including pedestrians suddenly crossing the road, vehicles suddenly stopping ahead, or road repairs.

[0066] Based on this, the embodiments of this application provide a simulation training processing method and system based on a heavy-duty cockpit. By combining high-fidelity simulation and data collection technology, it can provide intelligent driving training and evaluation for heavy-duty trucks (such as container trucks), thus solving the shortcomings of current technology in not being able to fully handle the complexity of heavy-duty truck driving.

[0067] Figure 1 is a structural block diagram of a simulation training processing system based on a heavy-duty cockpit according to an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0068] Referring to Figure 1, the system includes: a simulated heavy-duty cab 1, obtained from the cab of a real heavy-duty truck; a display screen 2, positioned around the simulated heavy-duty cab 1, which is a seamless, surround-type screen used to cover the entire driving field of view of the simulated heavy-duty cab 1 and display the simulated driving scenario; a data acquisition device 3, used to acquire raw driving training data corresponding to at least two rounds of simulated driving control operations performed by the user on the simulated heavy-duty cab 1 based on the simulated driving scenario; and a processor 4, used to process the raw driving training data corresponding to at least two rounds of simulated driving control operations to obtain target driving training data; and to analyze the target driving training data to obtain the user's training results, which are used to characterize whether there is a positive change in the user's driving control during at least two rounds of simulated driving control.

[0069] The simulated heavy-duty truck cab 1 can be derived from the cab of a real heavy-duty truck. By optimizing the dynamics of heavy-duty trucks to suit simulation requirements, it retains a complete structure and control system highly consistent with actual operation. In other words, the simulated heavy-duty truck cab 1 is completely identical to a real heavy-duty truck in terms of size, layout, operation, and driving feedback. This allows users to seamlessly match the spatial sense, control feedback, visual and tactile experience of a real heavy-duty truck when simulating driving, meeting the unique driving requirements of heavy-duty trucks. It accurately simulates complex operations such as steering, braking, and load control, ensuring that the assessment content is closely related to actual driving conditions. This enhances the driver's immersive and realistic operating experience, thereby significantly improving the realism and training value of simulated driving.

[0070] Unlike existing technologies that focus on the driving experience of light vehicles, this system takes into account the unique challenges of heavy-duty truck operation, such as longer braking distances and larger turning radii. It ensures that the system evaluates the driver based on real-world truck-specific parameters, providing more accurate driving behavior assessments that accurately reflect the driver's true operational capabilities.

[0071] For example, in order to optimize the weight and space utilization of the cab, unnecessary components unrelated to driver training (such as engine, transmission system, fuel tank) can be removed from the real heavy truck cab, while retaining all key components. For example, the simulated heavy truck cab 1 can be equipped with a steering wheel, brake pedal and accelerator pedal, and the simulated heavy truck cab 1 can be equipped with a left rearview mirror and a right rearview mirror. This allows the simulated heavy truck cab 1 to not only have the original vehicle-level driving environment, but also to be integrated with the display screen 2.

[0072] Display screen 2 can be a surround seamless screen, set around the simulated heavy-duty cockpit 1 to cover the entire driving field of vision of the simulated heavy-duty cockpit 1 and display the simulated driving scene; the specific structure and size of display screen 2 are not limited. For example, display screen 2 can be set around the simulated heavy-duty cockpit 1 to provide users with a 360-degree panoramic view to achieve full coverage of the driving field of vision, enhance the user's immersion, reduce the physical space occupied by the device, and thus improve the scalability and flexibility of the system. Display screen 2 can also be set in front of the simulated heavy-duty cockpit 1 and on the left and right sides according to the actual situation, so as to completely cover the driver's entire field of vision by using seamless screens on the front, left and right sides.

[0073] Furthermore, the display screen 2 can transition from a physical screen to holographic projection technology to provide a more immersive visual experience. For example, in this embodiment, virtual reality (VR) / augmented reality (AR) headsets, multi-projector systems, or dome-shaped screens can be used to replace the traditional three-sided screen wall to provide a more immersive and realistic driving experience. This can reduce physical space requirements or provide more seamless visual coverage, thereby enhancing driver training.

[0074] As an example, display screen 2 may include a seamlessly connected first display screen, second display screen, and third display screen. The first display screen can be used to display the simulated driving scene with a 180° field of view in front of the simulated heavy-duty truck cab 1. The second display screen can be matched with the left rearview mirror to display the simulated driving scene with a left-side field of view of the simulated heavy-duty truck cab 1. The third display screen can be matched with the right rearview mirror to display the simulated driving scene with a right-side field of view of the simulated heavy-duty truck cab 1. Based on this, by pairing display screen 2 with the rearview mirror, a more realistic driving environment can be simulated for the user more accurately, providing the driver with a comprehensive and seamless driving perspective, greatly enhancing the visual experience, especially suitable for simulated driving scenarios of heavy-duty trucks.

[0075] The data acquisition device 3 can be used to collect raw driving training data corresponding to at least two rounds of simulated driving control operations performed by the user on the simulated heavy-duty cab 1 based on simulated driving scenarios. The type of data acquisition device 3 is not limited; it can include high-precision sensors or other devices capable of collecting user training data. For example, the instrument panel of the simulated heavy-duty cab 1 can be enhanced to be configured with multiple high-precision sensors to collect various user operation data. Based on this, by configuring the data acquisition device 3, raw driving training data can be captured and analyzed in real time, more accurately simulating the unique response characteristics of heavy-duty trucks in steering, acceleration, braking, etc.

[0076] Processor 4 can be considered as a data processing unit of the simulation training processing system, and can implement the steps in any of the following method embodiments by executing a computer program. The computer program may include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form.

[0077] In a specific implementation, the simulation training processing system of this embodiment can be based on a multi-degree-of-freedom (such as 6-DOF or more) motion platform, which can move in real time with the driver's control of the steering wheel and gear lever to dynamically simulate the response of heavy trucks under complex road conditions, thereby improving the simulation accuracy and response capability of the simulation training processing system.

[0078] Figure 2 is a schematic diagram of a motion platform provided in an embodiment of this application. As shown in Figure 2, the motion platform can be composed of a support base (Bottom Plate), a connecting plate (Top Plate), and six electric push rods, which are used to provide roll, pitch, yaw, and translational movements in the X, Y, and Z directions. Based on this, it can ensure that the driver experiences realistic vehicle response, thereby improving the effectiveness of training.

[0079] Figure 3 is a schematic diagram of the architecture of a simulation training processing system provided in an embodiment of this application. As shown in Figure 3, the simulated heavy-duty driver's cab can be installed on the connection panel of the motion platform, which can ensure that the training environment is closely matched with the actual heavy-duty truck driving, thereby enhancing the realism of the driving experience.

[0080] Simultaneously, the front, left, and right sides of the simulated heavy-duty cab can be covered with seamless screen walls (i.e., a first display screen, a second display screen, and a third display screen) to provide a comprehensive view from the driver's seat. Building upon this, an immersive experience can be ensured by projecting the virtual environment onto the screens. Furthermore, a pre-stored space can be established between the screens and the heavy-duty truck cab to ensure that movement from the motion platform does not result in any collisions between the cab and the screens.

[0081] This embodiment provides a simulation training processing system based on a heavy-duty truck cab, comprising: a simulated heavy-duty truck cab derived from the cab of a real heavy-duty truck; a display screen, positioned around the simulated heavy-duty truck cab, which is a seamless, surround-type screen covering the entire driving field of view of the simulated heavy-duty truck cab and displaying a simulated driving scenario; a data acquisition device for acquiring raw driving training data corresponding to at least two rounds of simulated driving control operations performed by the user in the simulated heavy-duty truck cab based on the simulated driving scenario; and a processor for processing the raw driving training data corresponding to the at least two rounds of simulated driving control operations to obtain target driving training data; and for analyzing the target driving training data to obtain the user's training results, which characterize whether there is a positive change in the user's driving control during at least two rounds of simulated driving control. Using this system, by displaying the simulated driving scenario on a screen surrounding the simulated heavy-duty truck cab, and with the screen being a seamless, surround-type screen covering the entire driving field of view of the simulated heavy-duty truck cab, users can perform immersive simulated driving control operations based on the simulated driving scenario, providing heavy-duty truck drivers with the necessary visual experience and improving the effectiveness of heavy-duty truck simulated driving training.

[0082] Figure 4 is a flowchart illustrating a simulation training processing method based on a heavy-duty cockpit according to an embodiment of this application. It is provided as an example and not as a limitation. This method can be applied to a simulation training processing system based on a heavy-duty cockpit. As shown in Figure 4, the method includes:

[0083] S101, in response to the user's operation of opening the training cockpit in the simulated heavy-duty cockpit, displays the simulated driving scenario on the displays around the simulated heavy-duty cockpit.

[0084] The display screen is a wraparound seamless screen used to cover the entire driving field of vision of the simulated heavy-duty cockpit; the simulated driving scenario can be a virtual driving scenario provided to the user to evaluate the user's driving control. The content of the simulated driving field can be pre-configured according to actual needs, such as setting different scenarios according to the actual assessment content.

[0085] In some embodiments, the simulated driving scene includes at least a simulated driving environment. Before displaying the simulated driving scene on a display screen around the simulated heavy-duty cockpit, the method further includes: acquiring raw driving environment data, wherein the raw driving environment data is obtained by collecting outdoor driving environment data in at least one of different heights and different angles; performing three-dimensional reconstruction of the raw driving environment data using multi-view stereo reconstruction and point cloud generation technology to generate raw point cloud data of the raw driving environment data; and performing data processing on the raw point cloud data to obtain the simulated driving environment, wherein the data processing includes at least one of mesh generation processing, texture mapping processing, lighting adjustment processing, and format conversion processing.

[0086] In a specific implementation, the simulated driving scenario may include a simulated driving environment, which can refer to a road environment for users to drive. This environment can be constructed by performing a series of data processing steps in advance. For example, in order to enrich the driving scenario in the simulation training processing system and make it as realistic as possible, this embodiment can use street views captured by drones to construct a three-dimensional real-world road map. First, the outdoor driving environment can be collected by drones from at least one of different heights and angles to obtain raw driving environment data (such as high-resolution images). Three-dimensional reconstruction is performed using multi-view stereo reconstruction and point cloud generation technology to generate raw point cloud data of the original driving environment data. Subsequently, the raw point cloud data can be stitched, denoised, and geometrically corrected to ensure the integrity and accuracy of the data. Dynamic objects such as pedestrians and vehicles are removed using computer vision technology to avoid interfering with modeling.

[0087] Next, the generated point cloud data can be meshed, textured, and have its lighting adjusted to enhance the realism and detail fidelity of the 3D street scene model. Finally, after optimization and format conversion, the final simulated driving environment is obtained, ensuring smooth operation within the simulation training system. Based on this, by displaying a 3D real-world street scene map, a realistic simulated heavy truck driving environment is provided to users, greatly enhancing their immersion, spatial awareness, and interactivity. Simultaneously, efficient data processing technology improves modeling accuracy, visual consistency, and the usability of the virtual simulation. This embodiment is particularly suitable for evaluating the complex operation of heavy trucks, providing a more accurate representation of real-world driving conditions. This allows evaluators to more effectively observe and analyze driver behavior in complex road environments and dynamic traffic situations, resulting in more comprehensive and accurate evaluations.

[0088] In some embodiments, the simulated driving scenarios integrated into the driving simulator can not only simulate realistic road environments, but also incorporate dynamic traffic elements that require the driver to make appropriate reaction operations within a reasonable time frame.

[0089] For example, simulated driving scenarios can also include simulated blind spot areas and simulated dynamic events. Simulated blind spot areas can be considered blind spots simulated for heavy-duty driving cockpits. Simulated dynamic events can include at least dynamic traffic signal events and dynamic obstacle events. Dynamic traffic signal events can include events such as a yellow light appearing at an intersection or real-time changes in road sign signals. Dynamic obstacle events can be used to represent the presence of dynamic obstacles in the simulated driving environment, such as pedestrians or other vehicles suddenly appearing during driving, or sudden obstacles on the road. Based on this, by displaying diverse interactive virtual environments, the realism and effectiveness of training can be significantly enhanced. By simulating complex traffic conditions, especially blind spots caused by traffic signals and sudden events (such as pedestrians unexpectedly crossing the road or vehicles suddenly stopping ahead), the driver's emergency response capabilities and decision-making skills in emergency situations can be comprehensively assessed, providing a more comprehensive training experience and making the assessment more diverse and realistic, laying a scientific foundation for driver performance evaluation in complex scenarios.

[0090] In this embodiment, simulated dynamic events can be generated using predefined rules, randomization techniques, or artificial intelligence and machine learning to dynamically create more complex and personalized emergency scenarios. For example, AI-driven scene generators, real traffic footage, or predictive traffic simulation software can be used to create dynamic and realistic driving environments, enhancing the variability of the scenarios and allowing the system to adjust the difficulty in real time based on the driver's performance, thereby further improving training results.

[0091] S102. In response to at least two rounds of simulated driving control operations performed by the user on the simulated heavy-duty cockpit based on the simulated driving scenario, obtain the original driving training data corresponding to each round of simulated driving control operations.

[0092] Raw driving training data can be the original driving data that the user performs each round of simulated driving control operation. It can be understood as the original driving data collected by the data acquisition device. The type of raw driving training data can be determined according to the type of actual data acquisition device. Relevant personnel can pre-configure different data acquisition devices according to different actual needs.

[0093] After the simulated driving scenario is displayed on the screens around the simulated heavy-duty cockpit, the user can perform corresponding simulated driving control operations on the simulated heavy-duty cockpit based on the simulated driving scenario. For example, in this embodiment, the user can be prompted to conduct at least two rounds of simulated driving training. The simulated training processing system can respond to the user's at least two rounds of simulated driving control operations on the simulated heavy-duty cockpit based on the simulated driving scenario and acquire the original driving training data corresponding to each round of simulated driving control operations. The acquisition method can be, for example, by communicating and interacting with the data acquisition device to directly obtain the corresponding original driving training data, or by forwarding the corresponding original driving training data through other communication devices. This embodiment does not limit this.

[0094] S103. Process the original driving training data corresponding to at least two rounds of simulated driving control operations to obtain the target driving training data.

[0095] S104. Perform data analysis on the target driver training data to obtain the user's training results.

[0096] The training results are used to characterize whether there are positive changes in the user's driving control during at least two rounds of simulated driving control. Target driving training data can be understood as driving training data that significantly affects the evaluation results, and can be configured according to actual needs.

[0097] Specifically, this embodiment can process the original driving training data corresponding to at least two rounds of simulated driving control operations to obtain target driving training data. The data processing method is not limited. For example, the data processing process of the original driving training data can be implemented based on a neural network model, and the target driving training data can be directly output through the neural network model. Alternatively, the target driving training data can be obtained by taking a series of processing logics, such as screening and selecting the original driving training data, or performing certain calculation logic, etc. This embodiment does not limit the specific process of the above data processing, as long as the target driving training data can be obtained.

[0098] Finally, data analysis can be performed on the selected target driving training data to obtain the user's training results. For example, if the training program has a statistically significant main effect on driving performance (confidence level of 90%), it can prove the effectiveness of the training program, and the user's driving performance will generally improve over time.

[0099] As an example, if the standard deviation of lateral control and speed is significantly lower in the second round of driving training compared to the first two rounds, it indicates that the lateral and speed control capabilities have improved.

[0100] As another example, to examine the improvement in driver behavior, this embodiment can also use the analytic hierarchy process (AHP) to assign weights to different levels of driver behavior. According to this framework, after each round of driver training, the driver's behavior can be evaluated to obtain a corresponding comprehensive score. Subsequently, based on the results of the three rounds of comprehensive scores, a significance test can be performed using analysis of variance (ANOVA). The test results (i.e., the evaluation results) can then explain whether the driver's driving behavior has significantly improved after training.

[0101] This embodiment provides a simulation training processing method based on a heavy-duty truck cockpit. In response to a user's activation of the training operation on the simulated heavy-duty truck cockpit, a simulated driving scene is displayed on a screen surrounding the simulated heavy-duty truck cockpit. The screen is a seamless, surround-type display that covers the entire driving field of view of the simulated heavy-duty truck cockpit. In response to the user performing at least two rounds of simulated driving control operations on the simulated heavy-duty truck cockpit based on the simulated driving scene, raw driving training data corresponding to each round of simulated driving control operations is acquired. Data processing is performed on the raw driving training data corresponding to the at least two rounds of simulated driving control operations to obtain target driving training data. Data analysis is performed on the target driving training data to obtain the user's training results. The training results are used to characterize whether there is a positive change in the user's driving control during the at least two rounds of simulated driving control. Using this method, by displaying a simulated driving scene on a screen surrounding the simulated heavy-duty truck cockpit, and using a seamless, surround-type display that covers the entire driving field of view of the simulated heavy-duty truck cockpit, users can perform immersive simulated driving control operations on the simulated heavy-duty truck cockpit based on the simulated driving scene, providing the necessary visual experience for heavy-duty truck drivers and improving the effectiveness of heavy-duty truck simulated driving training.

[0102] Figure 5 is a flowchart illustrating a simulation training processing method based on a heavy-duty cockpit according to another embodiment of this application. In this embodiment, the original driving training data corresponding to at least two rounds of simulated driving control operations are processed to obtain target driving training data. The optimization is further as follows: the original driving training data of each round of simulated driving control operations in at least two rounds of simulated driving control operations are preprocessed to obtain preprocessed training data for each round of simulated driving control operations; the preprocessed training data corresponding to at least two rounds of simulated driving control operations are modeled to obtain initial generalized linear mixture models corresponding to at least two rounds of simulated driving control operations; a target generalized linear mixture model is selected from the initial generalized linear mixture models corresponding to at least two rounds of simulated driving control operations; and the target generalized linear mixture model is used as the target driving training data.

[0103] As shown in Figure 5, the method includes:

[0104] S201, In response to the user's operation of opening the training cockpit in the simulated heavy-duty cockpit, the simulated driving scenario is displayed on the screens around the simulated heavy-duty cockpit.

[0105] S202, in response to at least two rounds of simulated driving control operations performed by the user on the simulated heavy-duty cockpit based on the simulated driving scenario, acquire the original driving training data corresponding to each round of simulated driving control operations.

[0106] S203. Preprocess the original driving training data for each round of simulated driving control operation in at least two rounds of simulated driving control operation to obtain preprocessed training data for each round of simulated driving control operation.

[0107] As an executable implementation, acquiring the original driving training data corresponding to each round of simulated driving control operation may include at least one of the following: acquiring the original driving speed corresponding to each round of simulated driving control operation through a speed sensor configured in the simulated heavy-duty driving cockpit; acquiring the original lateral position data corresponding to each round of simulated driving control operation through a lane departure sensor configured in the simulated heavy-duty driving cockpit; acquiring the steering wheel angle value corresponding to each round of simulated driving control operation through a steering wheel sensor configured in the simulated heavy-duty driving cockpit; acquiring the braking speed and braking time corresponding to each round of simulated driving control operation through a pedal sensor configured in the simulated heavy-duty driving cockpit; acquiring fixation point data, blink frequency, and eye closure time corresponding to each round of simulated driving control operation through an eye tracker configured in the simulated heavy-duty driving cockpit; acquiring head direction data, gaze movement distance, and gaze movement time corresponding to each round of simulated driving control operation through an eye tracker configured in the simulated heavy-duty driving cockpit; and acquiring the applied accelerator pressure value corresponding to each round of simulated driving control operation through an accelerator pedal sensor configured in the simulated heavy-duty driving cockpit.

[0108] The original driving speed can refer to the real-time driving speed of the user during each round of simulated driving control operation, and the speed sensor may include, for example, a wheel rotation sensor; the original lateral position data can be used to characterize the lateral displacement of the simulated heavy-duty cockpit relative to the center of the lane during each round of simulated driving control operation, and the lane departure sensor may include a LiDAR (Light Laser Detection and Ranging) or lane departure sensor installed under the simulated heavy-duty cockpit; the steering wheel angle value can be used to characterize the real-time rotation angle of the steering wheel, and the type of steering wheel sensor may include an angle sensor; the type of pedal sensor is not limited, and may include a brake pressure sensor and / or a speed sensor, etc., the braking speed may refer to the deceleration during the braking process, and the braking time may be the time when the user actually presses the brake pedal.

[0109] The type of accelerator pedal sensor is not limited. It may include an accelerator pedal position sensor, i.e. a throttle position sensor, which can capture the position and movement state of the pedal. Its main functions may include sensing and transmitting the driver's operation state of the accelerator pedal to the processor, such as converting the applied accelerator pressure value and pedal movement speed, which are used to characterize the pedal force, into electrical signals.

[0110] The eye tracker in this embodiment is equipped with advanced data acquisition capabilities, enabling it to capture detailed eye movement data of the driver. For example, the eye tracker can use four depth cameras, and the data acquisition frequency can be 10Hz. Fixation point data can be used to analyze the driver's attention and focus. Fixation point recording can include the timestamp of the driver's gaze and eye movement coordinates, which can be understood as the coordinates of the user's fixation point on a two-dimensional screen. Blink frequency records the number of blinks per second; eye closure time is the total time the driver keeps their eyes closed. Head orientation data can be used to record the left-right rotation angle and up-down tilt angle of the head, in degrees. Eye movement distance can characterize the distance the user's gaze moves between different visual areas, and eye movement time is the time spent on eye movement.

[0111] At the same time, the eye tracker can also collect the user's gesture type, whether the hands are on the steering wheel, the 3D position of the left and right hands, and the 3D position of the steering wheel. All 3D positions are measured relative to the camera.

[0112] Furthermore, the accelerator pedal in this embodiment employs Hall effect precision technology for measurement, supports analog voltage input communication, and features high accuracy and durability. Its dimensions are 183mm in length and 103mm in width, with an input voltage of 5V and an output voltage range of 0-4.2V. It is suitable for operating environments with temperatures of 5℃-50℃ and humidity of 20%-80%. The accelerator pedal provides a stable analog voltage signal output and is compatible with chips for data transmission or control, making it suitable for acceleration control applications within a specific environmental range. Acceleration and deceleration can be used to evaluate the dynamic performance of a vehicle, and can be captured in real time using an acceleration sensor (e.g., an inertial measurement unit installed in the vehicle). The formula for calculating acceleration is... Where Δv is the change in velocity and Δt is the time interval.

[0113] Furthermore, based on the aforementioned raw driving training data, the following step will optimize the process of preprocessing the raw driving training data for each of the at least two rounds of simulated driving control operations to obtain preprocessed training data for each round of simulated driving control operations. This step may include at least one of the following:

[0114] Calculate the maximum driving speed and speed standard deviation for each round of simulated driving control operation based on the original driving speed; calculate the lateral position deviation based on the original lateral position data and lane center position data for each round of simulated driving control operation; calculate the angular velocity, angular acceleration, and angular direction data for each round of simulated driving control operation based on the steering wheel angle value; calculate the braking force and braking reaction time for each round of simulated driving control operation based on the braking speed; calculate the eye closure percentage for each round of simulated driving control operation based on the eye closure time; calculate the gaze transfer speed for each round of simulated driving control operation based on the head direction data, gaze movement distance, and gaze movement time; and calculate the throttle usage control rate for each round of simulated driving control operation based on the applied throttle pressure value.

[0115] In specific implementations, sensors, eye-tracking devices, and other monitoring equipment can be used to collect and analyze driver operation data in real time.

[0116] For example, in the process of simulated driver training, this embodiment can first collect driving behavior data of the driver in each round of simulated driving control operation under various scenarios, and then preprocess the original driving training data to obtain preprocessed training data for each round of simulated driving control operation. This can include: calculating the maximum driving speed and speed standard deviation corresponding to each round of simulated driving control operation. The maximum driving speed can refer to the highest speed reached by the driver during driving. The speed standard deviation can be a key indicator for evaluating the driver's speed control ability, used to measure the degree of speed fluctuation during driving, reflecting the driver's ability to maintain consistent and stable speed control. The formula for calculating the speed standard deviation can be... Among them, v i This represents the original driving speed for each recorded instance. This refers to the average speed over a given time period, where N is the number of speed data points recorded. Lateral position deviation data refers to the distance a vehicle deviates from its lane centerline during operation. The formula for calculating lateral position deviation data is: Among them, y i This represents the raw horizontal position data at each recording time, y center This indicates the center position data of the lane.

[0117] In this embodiment, the steering wheel angle sensor can collect or calculate information related to the steering wheel, such as steering wheel angle value, steering wheel angular velocity, steering wheel angular acceleration (i.e., steering wheel angle change rate), steering direction data (e.g., left or right turn), and steering wheel zero-point position. The steering wheel angle change rate reflects the driver's input amplitude during steering operations. In data analysis, the steering wheel angle change rate can be a commonly used indicator for evaluating driver steering behavior, and its calculation formula can be... Where Δθ is the change in steering wheel angle and Δt is the time interval.

[0118] Braking force can reflect the driver's braking response in emergency situations; the measured braking force F brake The calculation formula is F brake = m·adeceleration, where m is the mass of the vehicle and adeceleration is the deceleration during braking; braking reaction time can be defined as the time interval between when the driver sees or senses the need to brake and when the driver actually presses the brake pedal. It can be captured using the synchronization of the simulator's event trigger and the pedal sensor. The formula for braking reaction time is T. reaction =t brake -t stimulus , where t brake It is the point in time when the driver applies the brakes, t stimulus It simulates the point in time when a dynamic event (such as the appearance of an obstacle) occurs.

[0119] Eye movement speed measures a driver's ability to shift attention between different visual areas. The formula for eye movement speed is: Where, d eye It is the distance the line of sight moves, t saccade This refers to the time spent on eye movement; the percentage of time the eyes are closed can be a key indicator used in eye tracking to assess driver fatigue, representing the percentage of time a driver's eyelids are closed for more than 80% of the given time. The calculation formula is as follows: Among them, T closed It is the time the eyes are closed, T total This is the total observation time.

[0120] Furthermore, throttle usage reflects the driver's control over vehicle acceleration. Using a throttle pedal sensor, the pressure applied to the throttle and the duration of throttle use can be measured. The throttle control rate can be expressed as... Among them, P throttle It is the current pressure applied to the throttle (i.e., the applied throttle pressure value), P max It is the maximum pressure that the accelerator pedal can withstand.

[0121] Based on this, the simulation training processing method of this embodiment can collect a wide range of real-time data, including speed, lateral position, steering wheel angle, braking force, and eye-tracking data. This not only covers all aspects of driver performance, helping assessors to better understand driver behavior, but also, when combined with advanced analytics algorithms, can generate detailed assessment reports, providing assessors with in-depth analysis of driving operations and precise data support for subsequent personalized skills development, significantly improving the accuracy and scientific reliability of the assessment.

[0122] S204. Model the preprocessed training data corresponding to at least two rounds of simulated driving control operations to obtain the initial generalized linear hybrid model corresponding to at least two rounds of simulated driving control operations.

[0123] S205. Select a target generalized linear mixture model from the initial generalized linear mixture models corresponding to at least two rounds of simulated driving control operations.

[0124] S206. Use the target generalized linear mixture model as target driving training data.

[0125] S207. Perform data analysis on the target driver training data to obtain the user's training results.

[0126] The initial generalized linear mixture model can be understood as the modeling process performed on the preprocessed training data to obtain the initial generalized linear mixture model. The target generalized linear mixture model is the generalized linear mixture model obtained by screening and selecting the driving training data in the initial generalized linear mixture model.

[0127] In a specific implementation, driving simulation training may include at least two rounds, with the scenario and time of each round remaining consistent. To examine the impact of simulation training on driver behavior, after obtaining the preprocessed training data, a generalized linear mixed model (GLMM) can be used to evaluate the influence of training on driver behavior. For example, the preprocessed training data corresponding to at least two rounds of simulated driving control operations can be modeled. Specifically, the model can be built according to driving performance indicators as functions of driver attributes, traffic conditions, weather, training period, and other experimental control conditions, resulting in an initial GLMM corresponding to at least two rounds of simulated driving control operations.

[0128] Next, the driver training data in the initial generalized linear mixed model can be screened to obtain the target generalized linear mixed model. For example, stepwise backward regression can be used to select driver training data that meets preset parameter conditions from the initial generalized linear mixed model, thus obtaining the target generalized linear mixed model. The preset parameter conditions can be pre-defined selection rules, such as including driver training data that are statistically significant at the 5% level. The stepwise backward regression technique involves starting with a model containing all independent variables and sequentially eliminating independent variables. A hypothesis test is performed after each elimination to check whether the impact of elimination on the model is significant. If significant, the independent variable is eliminated; otherwise, it is retained. This process continues until no more variables can be eliminated or the preset stopping criteria are met.

[0129] The specific data selection process can begin with an initial generalized linear mixture model containing all driving training data. Driving training data that does not significantly affect the results can be gradually removed. The model can be re-estimated after each removal of driving training data until only driving training data that is statistically significant at the 5% significance level remains. That is, the influence of these driving training data is strong enough and unlikely to be randomly generated. Finally, a concise but effective target generalized linear mixture model is obtained. The target generalized linear mixture model contains only driving training data that is truly important to the results, and the obtained target generalized linear mixture model is used as the target driving training data.

[0130] Finally, the training results for users can be obtained by performing data analysis on the target driving training data. For example, the Akaike Information Criterion (AIC) and the Schwarz-Bayesian Criterion (SBC) can be used to evaluate the model's fit and thus obtain the user's training results.

[0131] This embodiment provides a simulation training processing method based on a heavy-duty cockpit. By modeling and processing the preprocessed training data corresponding to at least two rounds of simulated driving control operations, the accuracy of the target driving training data is improved, providing a precise data foundation for subsequent analysis to obtain training results.

[0132] As an alternative implementation, this embodiment can use biosensor technology, such as an electroencephalogram (EEG) sensor to monitor the user's brain activity in real time, and more accurately assess the driver's attention and reaction ability.

[0133] As an optional implementation, this embodiment can also integrate an AI-driven personalized training module. For example, by using machine learning algorithms to analyze the driver's performance data over time, the training scenario and difficulty level can be adjusted according to each driver's strengths and weaknesses. By providing customized feedback and training paths, the system can more effectively improve individual driver skills and ensure targeted development.

[0134] ii. Combine haptic feedback with augmented reality elements.

[0135] As an optional implementation, this embodiment can also add advanced haptic feedback devices combined with augmented reality elements. By integrating haptic technology into the steering wheel, pedals, and seat, the driver can receive haptic feedback, simulating the physical forces and vibrations experienced under real driving conditions, thereby enhancing the realism of the training. Furthermore, AR overlays can be used to directly overlay information (such as guidance prompts or real-time feedback) onto the driver's view, thereby enhancing the user's situational awareness and decision-making skills.

[0136] As can be seen from the above description, the simulation training processing method and system of this embodiment can be directly applied to the driver training and assessment of heavy truck operators. It is of practical significance for training drivers in complex operational scenarios simulating real-world conditions, including emergency response and decision-making in traffic congestion. Accordingly, the above-mentioned simulation training processing system can be applied to logistics companies, transportation training centers, and regulatory agencies to ensure that drivers meet specific safety and performance standards.

[0137] Alternatively, the aforementioned simulation training processing system can also be used for driver training of other specialized vehicles (such as construction machinery, military vehicles, or emergency response trucks), and can also be used by automakers for research and development to test driver interaction with new models and technologies before actual deployment, as well as as a tool for behavioral research related to driver safety and reaction analysis under controlled conditions.

[0138] Furthermore, the simulation training processing system in this embodiment significantly reduces the risks and costs associated with actual road assessments through virtual simulation, especially in terms of vehicle maintenance and fuel costs. The repeatability and stability of virtual simulation make the assessment more flexible, enabling multiple tests of dangerous maneuvers in a simulated environment without incurring actual risks. This is particularly effective for complex heavy truck operations, reducing the risk of potential accidents and making driver skill assessments safer and more efficient.

[0139] In summary, this embodiment relates to an intelligent driving training and assessment system specifically designed for heavy truck drivers. By employing extended reality technology, it fully simulates the complexity and dangers of real-world traffic scenarios, while simultaneously tracking driver behavior in real time to provide a highly realistic driving experience. Furthermore, by combining data analysis and intelligent assessment, it enhances the driver's emergency response capabilities.

[0140] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0141] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0142] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0143] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A heavy cockpit based simulation training processing method, wherein, The method comprises the following steps: displaying a simulated driving scene in a display screen around a simulated heavy-duty cockpit in response to a user's opening training operation on the simulated heavy-duty cockpit, wherein the display screen is a wraparound seamless screen for covering the entire driving view of the simulated heavy-duty cockpit; obtaining original driving training data corresponding to each round of simulated driving control operation of the user based on the simulated driving scene; performing data processing on the original driving training data corresponding to at least two rounds of simulated driving control operation to obtain target driving training data; performing data analysis on the target driving training data to obtain a training result of the user, wherein the training result is used to represent whether the user's driving control has positive changes during at least two rounds of simulated driving control.

2. The heavy cockpit based simulation training process method of claim 1, wherein, The simulated driving scene at least comprises a simulated driving environment, and before the simulated driving scene is displayed in the display screen around the simulated heavy-duty cockpit, the method further comprises the following steps: obtaining original driving environment data, wherein the original driving environment data is obtained by collecting outdoor driving environment in at least one of different heights and different angles; performing three-dimensional reconstruction on the original driving environment data by using multi-view stereoscopic reconstruction and point cloud generation technology to generate original point cloud data of the original driving environment data; performing data processing on the original point cloud data to obtain the simulated driving environment, wherein the data processing comprises at least one of grid division processing, texture mapping processing, illumination adjustment processing and format conversion processing.

3. The heavy cockpit based simulation training process method of claim 2, wherein, The simulated driving scene further comprises a simulated blind area and a simulated dynamic event, wherein the simulated dynamic event at least comprises a dynamic traffic signal event and a dynamic obstacle event.

4. The heavy cockpit based simulation training process method of claim 1, wherein, The data processing on the original driving training data corresponding to at least two rounds of simulated driving control operation to obtain target driving training data comprises the following steps: respectively pre-processing the original driving training data of each round of simulated driving control operation in at least two rounds of simulated driving control operation to obtain pre-processed training data of each round of simulated driving control operation; performing modeling processing on the pre-processed training data corresponding to at least two rounds of simulated driving control operation to obtain an initial generalized linear mixed model corresponding to at least two rounds of simulated driving control operation; selecting a target generalized linear mixed model from the initial generalized linear mixed model corresponding to at least two rounds of simulated driving control operation; taking the target generalized linear mixed model as the target driving training data.

5. The heavy cockpit based simulation training process method of claim 4, wherein, The selection of the target generalized linear mixed model from the initial generalized linear mixed model corresponding to at least two rounds of simulated driving control operation comprises the following steps: adopting stepwise backward regression technology to select driving training data meeting preset parameter conditions from the initial generalized linear mixed model to obtain a target generalized linear mixed model.

6. The heavy cockpit based simulation training process method of claim 4, wherein, The obtaining of the original driving training data corresponding to each round of simulated driving control operation comprises at least one of the following steps: The speed sensor configured in the simulated heavy-duty cockpit collects the original driving speed corresponding to each round of simulated driving control operation; The lane deviation sensor configured in the simulated heavy-duty cockpit collects the original lateral position data corresponding to each round of simulated driving control operation; The steering wheel sensor configured in the simulated heavy-duty cockpit collects the steering wheel angle value corresponding to each round of simulated driving control operation; The pedal sensor configured in the brake pedal in the simulated heavy-duty cockpit collects the brake speed and brake time corresponding to each round of simulated driving control operation; The eye tracker configured in the simulated heavy-duty cockpit collects the gaze point data, blink frequency and eye closure time corresponding to each round of simulated driving control operation; The eye tracker configured in the simulated heavy-duty cockpit collects the head direction data, line of sight movement distance and line of sight movement time corresponding to each round of simulated driving control operation; The accelerator pedal sensor configured in the simulated heavy-duty cockpit collects the applied accelerator pressure value corresponding to each round of simulated driving control operation.

7. The heavy cockpit based simulation training process method of claim 6, wherein, The preprocessing of the original driving training data of each round of simulated driving control operation in at least two rounds of simulated driving control operation includes at least one of the following: Based on the original driving speed corresponding to each round of simulated driving control operation, the highest driving speed and speed standard deviation corresponding to each round of simulated driving control operation are calculated; Based on the original lateral position data and lane center position data corresponding to each round of simulated driving control operation, the lateral position deviation data corresponding to each round of simulated driving control operation is calculated; Based on the steering wheel angle value corresponding to each round of simulated driving control operation, the steering angle speed, steering angle acceleration and steering angle direction data corresponding to each round of simulated driving control operation are calculated; Based on the brake speed corresponding to each round of simulated driving control operation, the braking force corresponding to each round of simulated driving control operation is calculated, and based on the brake time corresponding to each round of simulated driving control operation, the brake reaction time corresponding to each round of simulated driving control operation is calculated; Based on the eye closure time corresponding to each round of simulated driving control operation, the eye closure percentage corresponding to each round of simulated driving control operation is calculated; Based on the head direction data, line of sight movement distance and line of sight movement time corresponding to each round of simulated driving control operation, the line of sight transfer speed corresponding to each round of simulated driving control operation is calculated; Based on the applied accelerator pressure value corresponding to each round of simulated driving control operation, the throttle usage control rate corresponding to each round of simulated driving control operation is calculated.

8. The heavy cockpit based simulation training process method of any one of claims 1-7, wherein, The display screen comprises seamlessly connected first, second and third display screens, the first display screen is used to display the simulated driving scene in the 180° visual angle range in front of the simulated heavy-duty cab, the second display screen is matched with the left rearview mirror of the simulated heavy-duty cab and is used to display the simulated driving scene in the visual angle range on the left side of the simulated heavy-duty cab, and the third display screen is matched with the right rearview mirror of the simulated heavy-duty cab and is used to display the simulated driving scene in the visual angle range on the right side of the simulated heavy-duty cab.

9. A heavy cockpit based simulation training processing system, wherein, Comprise: a simulated heavy-duty cab obtained by a real heavy-duty truck cab; a display screen arranged around the simulated heavy-duty cab, the display screen being a surrounding seamless screen used to cover the entire driving visual field of the simulated heavy-duty cab and display a simulated driving scene; a data acquisition device used to acquire original driving training data corresponding to at least two rounds of simulated driving control operations of the simulated heavy-duty cab based on the simulated driving scene; a processor used to perform data processing on the original driving training data corresponding to the at least two rounds of simulated driving control operations to obtain target driving training data; perform data analysis on the target driving training data to obtain a training result of the user, the training result being used to represent whether the driving control of the user has a positive change in the at least two rounds of simulated driving control.

10. The heavy cockpit-based simulation training processing system of claim 9, wherein, The simulated heavy-duty cab is provided with a steering wheel, a brake pedal and an accelerator pedal inside, and is provided with left and right rearview mirrors outside.

11. The heavy cockpit-based simulation training processing system of claim 10, wherein, The display screen comprises seamlessly connected first, second and third display screens, the first display screen is used to display the simulated driving scene in the 180° visual angle range in front of the simulated heavy-duty cab, the second display screen is matched with the left rearview mirror of the simulated heavy-duty cab and is used to display the simulated driving scene in the visual angle range on the left side of the simulated heavy-duty cab, and the third display screen is matched with the right rearview mirror of the simulated heavy-duty cab and is used to display the simulated driving scene in the visual angle range on the right side of the simulated heavy-duty cab.

Images