Multi-interface interactive training simulation method and system, medium and program product

Abstract

The invention discloses a multi-interface interactive training simulation method and system, a medium and a program product, and relates to the technical field of simulation training, and the method comprises the steps: determining a target training subject according to the selection of a student, determining a standard training procedure, displaying a virtual operation panel corresponding to the target training subject on a first display screen, a working principle animation corresponding to the target training subject is displayed on a second display screen, and a real-time simulation state corresponding to the target training subject is displayed on a third display screen; acquiring operation data of a student on the virtual operation panel in real time, matching the operation data with an operation step sequence in a standard training procedure, generating an operation instruction sequence, driving a working principle animation of the second display screen and a real-time simulation state of the third display screen, and recording actual execution time and operation deviation data; and inputting to a preset scoring model to obtain a final operation score, and generating a training report. According to the invention, pertinence of the trainee in simulated driving training can be improved.

Description

Technical Field

[0001] This application relates to the technical field of simulation training, and particularly to a multi-interface interactive training simulation method, system, medium and program product. Background Art

[0002] With the growth of the number of vehicles and the expansion of driver training needs, driving training institutions need to provide more efficient and safer training methods for trainees. Traditional driving training mainly relies on in-vehicle training. This method not only requires a large investment in vehicles and site resources, but also faces the influence of various uncertain factors such as weather and road conditions, making it difficult to ensure the continuity and stability of training.

[0003] In related technologies, the training methods adopted by driving training institutions mainly include in-vehicle training and simple simulator training. In-vehicle training is that a coach guides trainees to directly operate real vehicles in a training ground to learn various basic driving skills such as straight driving, turning, and reversing; simple simulator training is to cooperate with computer software through basic hardware devices such as steering wheels and pedals to let trainees perform basic driving operation exercises in a fixed scenario.

[0004] However, the training methods in related technologies lack quantitative training evaluation and are difficult to record data on the proficiency of trainees' training, often resulting in poor training effects. Summary of the Invention

[0005] This application provides a multi-interface interactive training simulation method, system, medium and program product for improving the pertinence of trainees' simulated driving training.

[0006] In a first aspect, this application provides a multi-interface interactive training simulation method applied to a simulation training system. The method includes: determining a target training subject according to the trainee's selection, and retrieving a corresponding standard training procedure from a training subject library; according to the standard training procedure, displaying a virtual operation panel corresponding to the target training subject on a first display screen, displaying an animation of the working principle corresponding to the target training subject on a second display screen, and displaying a real-time simulation state corresponding to the target training subject on a third display screen; collecting operation data of the trainee on the virtual operation panel in real time, matching the operation data with the operation step sequence in the standard training procedure to generate an operation instruction sequence; synchronously driving the animation of the working principle on the second display screen and the real-time simulation state on the third display screen according to the operation instruction sequence, and recording the actual execution time and operation deviation data of each operation step in the operation instruction sequence; inputting the actual execution time and operation deviation data into a preset scoring model to obtain a final operation score; generating a training report including error-prone step analysis, operation time-consuming analysis and improvement suggestions according to the final operation score and operation deviation data.

[0007] In the above embodiments, the simulation training system realizes a comprehensive training effect evaluation by displaying a virtual operation panel, a working principle animation, and a real-time simulation status on three display screens respectively, in combination with operation data collection and a scoring model, enabling trainees to master the operation essentials, understand the working principle, and obtain timely feedback simultaneously, thereby improving the pertinence and efficiency of training.

[0008] In combination with some embodiments of the first aspect, in some embodiments, before the step of determining a target training subject according to the trainee's selection and retrieving the corresponding standard training procedure from the training subject library, the method further includes: receiving a login request from the trainee and verifying the trainee's identity information according to the user database; after the identity verification is passed, obtaining the trainee's historical training data; analyzing the trainee's training weak points based on the historical training data, and performing difficulty grading and priority ranking on the training subjects in the training subject library; the difficulty grading is calculated according to the average completion time, average number of errors, and average score in the historical training data, and the priority is determined according to the product of the difficulty level of the training subject and the lowest score of the corresponding subject in the trainee's historical training; recommending training subjects to the trainee in descending order of priority, and marking the difficulty level and estimated training time on the display interface of each training subject.

[0009] In the above embodiments, the simulation training system performs difficulty grading and priority ranking by analyzing the trainee's historical training data, provides personalized training recommendations for the trainee, makes the training plan more in line with the actual level of the trainee, and avoids the problem of low learning efficiency caused by the training content being too difficult or too easy.

[0010] In combination with some embodiments of the first aspect, in some embodiments, after the step of real-time collecting the operation data of the trainee on the virtual operation panel, matching the operation data with the operation step sequence in the standard training procedure, and generating an operation instruction sequence, the method further includes: when a key operation step is detected, converting the normal simulation mode to an intensive teaching mode; the key operation step is an operation step with an error rate exceeding a preset error threshold in the historical training data; in the intensive teaching mode, suspending the current training process, displaying the execution process of the current operation step in the form of action replay in the working principle animation on the second display screen, and simultaneously displaying the standard operation demonstration and the actual operation comparison screen in a split-screen manner on the third display screen; after detecting the trainee's confirmation of the learning completion instruction, converting the intensive teaching mode to the normal simulation mode and resuming the normal training process.

[0011] In the above embodiments, the simulation training system starts the intensive teaching mode for key operation steps with a relatively high historical error rate, strengthens learning through action replay and split-screen comparison, and ensures that trainees can accurately master the key and difficult operations.

[0012] In some embodiments in combination with some embodiments of the first aspect, after the steps of determining a target training subject according to the trainee's selection and retrieving a standard training procedure corresponding to the target training subject from a training subject library, the method further includes: determining a plurality of environmental factors based on an actual training scenario and constructing a dynamic simulation environment; the environmental factors include weather changes, light changes, and road condition changes; obtaining a proficiency index of the trainee and adjusting the environmental complexity of the dynamic simulation environment according to the proficiency index; after the trainee completes the standard training subject and selects a complex additional training option, generating a composite training scenario based on random environmental factors; the random environmental factors are one or more of the plurality of environmental factors; generating additional scoring items based on the composite training scenario.

[0013] In the above embodiments, the simulation training system constructs a dynamic simulation environment including weather, light, and road condition changes, and adjusts the environmental complexity according to the trainee's proficiency, making the training scenario closer to the actual driving situation and improving the practicality of the training.

[0014] In some embodiments in combination with some embodiments of the first aspect, the step of generating a composite training scenario based on random environmental factors after the trainee completes the standard training subject and selects a complex additional training option specifically includes: after the trainee completes the standard training subject and selects a complex additional training option, displaying road type characteristic options including urban roads, highways, and mountain roads; determining a target road type among the road type characteristic options based on the user option selection; simulating and generating a composite training scenario according to the target road type, the dynamic simulation environment, and the random environmental factors.

[0015] In the above embodiments, the simulation training system provides different road type options such as urban roads, highways, and mountain roads, and generates a composite training scenario in combination with dynamic environmental factors, enabling the trainee to adapt to various road condition environments.

[0016] In some embodiments in combination with some embodiments of the first aspect, the step of synchronously driving the working principle animation of the second display screen and the real-time simulation state of the third display screen according to an operation instruction sequence and recording the actual execution time and operation deviation data of each operation step in the operation instruction sequence specifically includes: calculating the dynamic working state parameters of each vehicle component in real time based on the operation instruction sequence and a vehicle dynamics model; simulating and generating a damping characteristic curve of the clutch according to the engine speed, vehicle speed, and road surface condition, and displaying the dynamic working state parameters and the damping characteristic curve on the third display screen; triggering a warning prompt when it is detected that an incorrect operation causes an abnormal vehicle working state, and displaying the stress distribution of the affected components on the second display screen; recording the actual execution time and operation deviation data of each operation step in the operation instruction sequence.

[0017] In the above embodiments, the simulation training system enables trainees to deeply understand the vehicle working principle and correct operation essentials by calculating vehicle dynamic working state parameters in real time, simulating the clutch damping characteristics, and displaying the component stress distribution in case of abnormal operation.

[0018] In combination with some embodiments of the first aspect, in some embodiments, after the step of recording the actual execution time and operation deviation data of each operation step in the operation instruction sequence, the method further includes: collecting the gaze tracking data and body posture data of the trainee during the operation; determining the safety awareness index including observing the rearview mirror and blind spot inspection according to the gaze tracking data; determining the posture standard index according to the body posture data; and identifying potential dangerous driving behaviors according to the safety awareness index and the posture standard index.

[0019] In the above embodiments, the simulation training system effectively identifies potential dangerous driving behaviors by collecting the gaze tracking and body posture data of the trainee, analyzing the safety awareness index and the posture standard index, and improves the safety and standardization of the training.

[0020] In a second aspect, an embodiment of the present application provides a simulation training system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the simulation training system to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product containing instructions, which, when the computer program product runs on the simulation training system, enables the simulation training system to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, which, when the instructions run on the simulation training system, enable the simulation training system to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0023] It can be understood that the simulation training system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Due to the adoption of a three-screen linkage display system, the virtual operation panel, the working principle animation, and the real-time simulation status are respectively displayed on different display screens, and training evaluation is carried out through real-time data collection and a scoring model. Therefore, it is possible to enable trainees to obtain intuitive feedback on the operation interface, in-depth understanding of the internal principle, and real-time grasp of the overall status during the operation process, effectively solving the problems of incomplete learning effects and unclear evaluation criteria caused by single-interface display in related technologies. Furthermore, it realizes the unity of visualization, principle transparency, and evaluation quantification in the training process, improving the training effect and learning efficiency.

[0025] 2. Due to the adoption of a dynamic simulation environment construction mechanism, environmental factors such as weather changes, light changes, and road condition changes are integrated into the training system, and the environmental complexity is dynamically adjusted according to the trainees' proficiency. Therefore, it is possible to provide trainees with gradually increasing training difficulty and a driving scenario closer to reality, effectively solving the problems of single training scenarios and lack of practicality in related technologies. Furthermore, it realizes the dynamic adaptation and real simulation of the training environment, greatly improving the practicality and pertinence of the training.

[0026] 3. Due to the adoption of a real-time calculation and monitoring mechanism based on the vehicle dynamics model, comprehensive analysis is carried out on parameters such as engine speed, vehicle speed, and road surface conditions, and visual display is carried out through the clutch damping characteristic curve and component stress distribution. Therefore, it is possible to enable trainees to intuitively understand the relationship between operations and vehicle status, effectively solving the problems that trainees in related technologies are difficult to understand the vehicle working principle and are prone to incorrect operations. Furthermore, it realizes the precise monitoring and timely correction of the training process, improving the safety and reliability of the training. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flowchart of a multi-interface interactive training simulation method in an embodiment of the present application; Figure 2 is a virtual structure diagram of a simulation training system in an embodiment of the present application; Figure 3 is another flowchart of a multi-interface interactive training simulation method in an embodiment of the present application; Figure 4 is a schematic structural diagram of a physical device of a simulation training system in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application, the singular forms "a", "an", "the above", "the", and "this" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations including one or more of the listed items.

[0029] Hereinafter, the terms "first" and "second" are only for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0030] For ease of understanding, the application scenarios of the embodiments of the present application are introduced below.

[0031] With the rapid growth of the demand for driving training, the traditional driving school training model faces many challenges. A large driving school in a certain city needs to train more than 5,000 students every year. However, due to factors such as the number of training vehicles, training grounds, and weather conditions, students often need to wait for a long time to conduct on-road training. At the same time, due to the lack of a standardized training evaluation system, coaches cannot accurately grasp the training progress and skill mastery of students, resulting in uneven training effects. Especially in complex environments such as rainy, snowy, or night conditions, there are significant safety hazards in on-road training, seriously affecting the continuity and efficiency of training.

[0032] In the related art, a simple simulator with a display screen and basic hardware devices can be used to practice basic driving operations. Such simulators usually only have a 42-inch display screen, a steering wheel, and a pedal assembly. Due to the lack of functions for demonstrating the working mechanism of the vehicle and collecting and analyzing training data, students can only master the operation essentials through repeated attempts, resulting in low training efficiency and lack of targeted guidance. The scenario of using the multi-interface interactive training simulation method in the related art is introduced below.

[0033] Currently, the driving simulators on the market mainly adopt the solution of a single display screen combined with basic hardware devices such as a steering wheel and pedals. For example, a certain brand of driving simulator can only display simple road scenes, lacking an intuitive display of the vehicle's working principle, and trainees cannot deeply understand the operation essentials. The simulation scenarios are single and cannot dynamically adjust the training difficulty according to the trainees' levels. At the same time, the simulator cannot record and analyze the trainees' operation data in real time, and the coach can only subjectively evaluate the trainees' performance based on experience. In practical applications, such simple simulation training often becomes a mere formality and is difficult to cultivate the trainees' actual driving skills.

[0034] However, by adopting the multi-interface interactive training simulation method in the embodiments of the present application, through the training mode of three-screen linkage, the synchronous display of the virtual operation panel, the vehicle working principle animation, and the real-time driving scene is realized. The system can not only collect and analyze in real time detailed training data including operation timing, force, trajectory, etc., but also automatically adjust the training difficulty and content according to the trainees' historical performance. Especially in key operation links such as clutch control and gear shifting, the system intuitively shows the relationship between the operation and the vehicle response through the working principle animation to help trainees quickly master the correct operation method. The following introduces the scenario using the multi-interface interactive training simulation method in the present application.

[0035] By adopting the multi-interface interactive training simulation system of the present application, the driving school can provide personalized training plans for each trainee. The system shows the virtual operation panel, the vehicle working principle animation, and the real-time simulation status respectively through three display screens to help trainees comprehensively understand the driving essentials. For example, when practicing the starting operation, the first screen shows the instrument panel and console in real proportion, the second screen dynamically shows the working principle of the clutch engagement process, and the third screen gives real-time feedback on the vehicle state changes. The system will also automatically identify error-prone points based on the trainees' historical training data and enter the intensive teaching mode to help trainees correct wrong actions through action replay and standard demonstrations.

[0036] It can be seen that by adopting the multi-interface interactive training simulation method in the embodiments of the present application, while realizing standardized training, it can also effectively solve problems such as lagging feedback and lack of pertinence in traditional training. Through the data-driven evaluation system, the system can accurately quantify the trainees' operation performance, identify weak training items, and provide targeted intensive training plans, thereby achieving the training goals of teaching students in accordance with their aptitude and continuous improvement.

[0037] For ease of understanding, the following combines the above scenario to describe the process of the method provided in this embodiment. Please refer to Figure 1 , which is a schematic flowchart of the multi-interface interactive training simulation method in the embodiments of the present application.

[0038] S101. Determine the target training subject according to the trainee's selection, and retrieve the corresponding standard training procedure from the training subject library.

[0039] Among them, the target training subject represents the specific driving operation items to be trained, such as gear operation, steering control, etc.; the training subject library refers to the database storing various driving training items, including the basic information, difficulty level, and training requirements of the training subjects, etc.; the standard training procedure represents the standardized operation process and evaluation criteria formulated for specific training subjects.

[0040] After the trainee logs in to the system and enters the training subject selection interface, the simulation training system executes this step. Specifically, the system first displays the list of optional training subjects. After the trainee selects the target training subject, the system retrieves the corresponding standard training procedure from the training subject library according to the subject number or name. The retrieved standard training procedure contains the complete information required for training, such as the sequence of operation steps, standard execution time, evaluation criteria, etc.

[0041] In some embodiments, the selection of training subjects and the retrieval of procedures can be achieved in various ways: optionally, the system constructs a training ability model based on the trainee's historical training data, analyzes the trainee's weak links through this model, and automatically recommends suitable training subjects in combination with the training progress. After the trainee confirms, the system retrieves the corresponding procedure; optionally, the system provides a multi-dimensional retrieval function for training subjects. The trainee can screen subjects by conditions such as difficulty level, training duration, operation type, etc. After selection, the system automatically matches the corresponding procedure and performs difficulty adaptation. It can be understood that other methods can also be used to achieve the intelligent matching of subject selection and procedure retrieval, which is not limited here.

[0042] S102. According to the standard training procedure, display the virtual operation panel corresponding to the target training subject on the first display screen, display the working principle animation corresponding to the target training subject on the second display screen, and display the real-time simulation state corresponding to the target training subject on the third display screen.

[0043] Among them, the virtual operation panel represents an interactive display interface that simulates the real vehicle operation interface, including operation elements such as a steering wheel, a gear shifter, and pedals; the working principle animation refers to a dynamic demonstration that shows the working states and internal mechanisms of various vehicle components; the real-time simulation state represents the vehicle running state in the current training scenario, including parameters such as vehicle speed and engine speed; the first, second, and third display screens are respectively used to display training information in different dimensions.

[0044] The simulation training system executes this step after completing the loading of training subjects and procedures. Specifically, the system first determines the operating elements and display parameters required for the current training subject according to the standard training procedures, and generates a corresponding virtual operation panel on the first display screen; at the same time, it loads the animation materials of the component working principles related to this training subject on the second display screen, and initializes the training scenario and vehicle status parameter display interface on the third display screen. The system ensures that the content of the three display screens is updated synchronously to provide trainees with all-round training feedback.

[0045] In some embodiments, the coordination of multi-interface display can be achieved in multiple ways: Optionally, the system adopts distributed display technology, and the main control module uniformly schedules the display content of the three display screens, automatically switches the animation scenes according to the operation progress, and synchronously updates the status data in real time to ensure the coherence of the display content; Optionally, the system is based on scene adaptive technology, dynamically adjusts the layout and display method of the display content for different training subjects, highlights the key display areas during key operation steps, and provides auxiliary display functions such as picture-in-picture. It can be understood that other ways can also be used to achieve the intelligent coordination of multi-interface display, which is not limited here.

[0046] The overall architecture of the simulation training system in the embodiments of the present application will be described below. Please refer to Figure 2 , which is the virtual structure schematic diagram of the simulation training system in the embodiments of the present application.

[0047] As Figure 2 shown in (a), the core of the simulation training system consists of three main subsystems: a virtual operation subsystem, a process control and principle display subsystem, and a training management subsystem. These three subsystems are implemented through the Figure 2 hardware architecture shown in (b). Specifically, the system adopts the form of "one machine with three screens", that is, one control host is configured with three display screens.

[0048] Among them, the first display screen is used for the simulation operation panel, which can simulate and display the actual operation panel, including mechanical, electronic devices or components that need to be actually operated, such as buttons, switches, manipulators, joysticks, etc. Trainees complete the control of the virtual panel through the mouse and keyboard, and can zoom in and out of the virtual panel according to the operation content. The second display screen is mainly used to display the linkage information of the trainees' operations, including the signal flow, operation logic, and 2D or 3D animations of the working principles during the operation process. The system uses this display screen to display the operation process, working principle, information flow, etc. in multiple ways such as text, voice prompts, and three-dimensional animations, and reflects the internal process logic and working principle in real time. The third display screen is used for training subject setting, training initiation, training assessment, and training result statistical analysis. Among them, the training subject setting and training initiation require trainees to manually operate and configure, while the training assessment and result statistical analysis are automatically completed by the system.

[0049] These three display screens are connected to the control host via HDMI cables and are also connected to peripheral speakers. The host not only runs the core software of the simulation training system but also is responsible for running the training management software. With this hardware configuration, the system achieves the effect of "one instruction, one display". When the trainee operates on the first display screen, the second and third display screens will display relevant operation feedback in various intuitive ways such as animations, voices, and texts.

[0050] It should be noted that the three display screens and the corresponding control system interface described in this embodiment should not be regarded as specific limitations to the solution. During the actual implementation process, the number of display screens, the display corresponding positions of each display screen, the installation method of the display screens, the functions reflected, etc. are not limited.

[0051] In addition, as Figure 2 shown in (c), in order to support the above functions, the system is divided into four layers in the software architecture: the basic hardware layer, the model data layer, the engine function layer, and the application platform layer. The hardware layer includes underlying supports such as workstations and basic network devices; the data layer stores resources such as animation materials, 3D models, and operation procedures required for training; the function layer contains a scene generation engine, a process simulation engine, and a training task engine, which are responsible for virtual scene generation, process principle display, and training task organization respectively; the application layer provides the final interaction interface.

[0052] This layered architecture realizes internal logical control through software to ensure that the operation interface, logical process, response phenomenon, and interaction logic are consistent with the actual situation. The system can either run independently or be used in combination with other simulation training equipment. In the independent operation mode, the trainee operates the virtual terminal, and the other two screens are linked for display; in the combined use mode, the virtual terminal can synchronously display the operation actions of the simulation training station, and the other two screens display the corresponding process principles and status information.

[0053] S103. Real-time collect the operation data of the trainee on the virtual operation panel, match the operation data with the operation step sequence in the standard training procedure, and generate an operation instruction sequence.

[0054] Among them, the operation data represents the digital record of various operation behaviors of the trainee on the virtual operation panel, including information such as operation type, intensity, and time; the operation step sequence refers to the standard operation process specified in the standard training procedure; the operation instruction sequence represents the instruction set generated after matching the trainee's actual operation with the standard steps.

[0055] The simulation training system executes this step in real time after starting the training mode. Specifically, the system collects all the operation behaviors of the trainee on the virtual operation panel in real time through sensors, converts these operations into standardized digital signals; then matches these signals with the operation steps defined in the standard training procedures in real time to analyze the correctness and timing relationship of the operations; finally, converts the matching results into an operation instruction sequence recognizable by the system for driving subsequent animation display and status update.

[0056] In some embodiments, the acquisition and matching of operation data can be implemented in various ways: Optionally, the system uses multi-modal sensing technology to collect operation data, combines deep learning algorithms to recognize operation intentions, and processes operation sequences through fuzzy matching algorithms to achieve fault tolerance processing for non-standard operations; Optionally, the system tracks the operation process based on a real-time state machine model and realizes fast and accurate operation matching by establishing a mapping relationship between operation feature vectors and standard steps. It can be understood that other ways can also be used to implement intelligent acquisition and matching processing of operation data, which is not limited here.

[0057] S104. Synchronously drive the working principle animation of the second display screen and the real-time simulation state of the third display screen according to the operation instruction sequence, and record the actual execution time of each operation step in the operation instruction sequence and the operation deviation data.

[0058] Among them, synchronous drive means coordinating and updating the content of multiple display interfaces according to operation instructions; the actual execution time refers to the actual time required for the trainee to complete each operation step; the operation deviation data represents the difference between the trainee's operation and the standard operation, including time difference, position deviation, etc.

[0059] The simulation training system executes this step immediately after generating the operation instruction sequence. Specifically, the system calculates the change of vehicle state parameters according to the operation instruction sequence, synchronously updates the working principle animation of relevant components on the second display screen to display the motion state of the internal mechanism; at the same time, the vehicle attitude, speed and other simulation state information are updated in real time on the third display screen. The system also records the execution time of each operation step, calculates various deviation data during the operation process, and gives a prompt on the third display screen.

[0060] In some embodiments, synchronous drive and data recording can be implemented in various ways: Optionally, the system uses a real-time rendering engine to process animation effects, simulates vehicle states based on a physics engine, ensures the coherence of the display content through a time synchronization mechanism, and at the same time establishes a complete operation log recording system; Optionally, the system uses a state prediction model to achieve smooth transition of the display content, processes operation delays through a data caching mechanism, and realizes efficient state synchronization. It can be understood that other ways can also be used to achieve precise control of interface synchronization and data recording, which is not limited here.

[0061] S105. Input the actual execution time and operation deviation data into a preset scoring model to obtain the final operation score.

[0062] The preset scoring model is a mathematical model used to evaluate the operation level of trainees, including multiple scoring dimensions and weight coefficients; the final operation score refers to the comprehensive score calculated based on various evaluation indicators.

[0063] The simulation training system executes this step after completing the training subject. Specifically, the system inputs the data such as the recorded actual execution time and operation deviation into the preset scoring model. This model comprehensively considers multiple dimensions such as operation accuracy, time efficiency, and movement smoothness, and calculates the final score reflecting the actual operation level of the trainee through weighted calculation.

[0064] It should be noted that the preset scoring model is trained using a historical training data set during training, and the input includes the standard operation step sequence and the actual operation data (execution time and operation deviation) of a large number of trainees. By comparing the differences between the standard operation and the actual operation, a scoring standard is established. The training criteria of the preset scoring model include multiple dimensions such as the accuracy of operation timing (the execution order and time interval of each step), the rationality of operation force (such as the control force of the clutch and accelerator), and the smoothness of operation trajectory (such as the coherence of steering wheel turning). The preset scoring model adopts a multi-layer neural network structure, takes the operation sequence features as the input layer, extracts operation features through multiple hidden layers, and finally outputs the operation score. The model includes a timing analysis module (evaluating the sequence of operation steps), a deviation calculation module (quantifying the difference between the actual operation and the standard operation), and a comprehensive scoring module (calculating the final score based on the evaluation results of different dimensions). By receiving the operation data stream of trainees in real time, including the actual execution time and operation deviation data of each operation step (such as the clutch engagement speed, braking force, steering angle, etc.). The preset scoring model can analyze these data in real time, output a standardized score of 0 - 100, and generate a specific description of the deduction items. At the same time, the model will also identify the key error operation points to provide a basis for subsequent intensive training.

[0065] S106. Generate a training report including error-prone step analysis, operation time-consuming analysis, and improvement suggestions based on the final operation score and operation deviation data.

[0066] Among them, the error-prone step analysis represents the statistics and analysis of the operation links where trainees are prone to make mistakes; the operation time-consuming analysis refers to the detailed analysis of the time consumption of each operation step; the improvement suggestions represent the targeted training suggestions given based on the analysis results.

[0067] The simulation training system executes this step after calculating the final score. Specifically, the system comprehensively analyzes the final operation score and various deviation data, identifies the weak links and error-prone points of the trainee during the training process, statistically analyzes the operation time distribution, and finally generates a training report containing detailed analysis content and targeted suggestions according to the analysis results.

[0068] In some embodiments, the generation of the training report can be achieved in various ways: Optionally, the system uses data mining techniques to analyze the operation data, identifies typical error patterns through clustering algorithms, and generates personalized improvement suggestions in combination with the expert knowledge base; Optionally, based on natural language generation techniques, the system converts the data analysis results into an easy-to-understand text description and realizes the automatic generation of the report through templatization and dynamic filling. It can be understood that other methods can also be used to achieve the intelligent generation and personalized customization of the training report, which is not limited here.

[0069] In the above embodiments, the system establishes a complete training file for the trainee, including detailed records such as operation data, error types, and correction situations of each training. In practical applications, these data are not only used to evaluate the training effect of the trainee, but also used to optimize the training content and difficulty. For example, when the system finds that the error rate of a certain type of operation is generally high, it will automatically increase the relevant intensive training content and adjust the training progress to ensure that the trainee truly masters the essentials before entering the next stage of training. The following supplements the scenario of this embodiment.

[0070] With the continuous optimization of the system, the driving school has further expanded the depth and breadth of the training content. After the basic training is completed, the system will automatically generate a composite training scenario including various weather conditions, light changes, and road condition characteristics according to the proficiency of the trainee. For example, the system can simulate high-difficulty subjects such as mountain road curves in rainy days and overtaking on the highway at night. At the same time, through computer vision technology, the line-of-sight trajectory and body posture data of the trainee are collected in real time, and their safety awareness and driving habits are analyzed to detect and correct potential bad driving behaviors early. This all-round training system improves the actual driving skills and safety awareness of the trainee.

[0071] After combining the above scenarios, the following further describes the more specific process of the method provided in this embodiment. Please refer to Figure 3 , which is another process schematic diagram of the multi-interface interactive training simulation method in the embodiment of the present application.

[0072] S301. Receive the login request of the trainee and verify the identity information of the trainee according to the user database.

[0073] Among them, the login request represents the identity authentication application submitted by the trainee through the system login interface; the user database refers to the data storage system that stores the trainee's identity information, account information, and permission configuration; the identity information represents various types of data used to verify the trainee's identity, including the account, password, personal information, etc.

[0074] The simulation training system executes this step when it detects that the trainee initiates a login operation. Specifically, the system first receives the login credentials entered by the trainee, and then accesses the user database for identity verification. The verification process includes account validity check, password matching verification, permission level confirmation, etc., to ensure that only legitimate users can enter the system.

[0075] In some embodiments, identity verification can be achieved in various ways: Optionally, the system adopts a multi-factor authentication mechanism, combines biometric recognition, dynamic passwords and other technologies to enhance security, and transmits authentication information through an encrypted channel to achieve the reliability of identity verification; Optionally, the system is based on the token authentication mechanism, and processes the login request through a distributed identity authentication service to achieve unified authentication for multiple terminals and single sign-on. It can be understood that other methods can also be used to achieve secure and reliable identity authentication, which is not limited here.

[0076] S302. After the identity verification is passed, obtain the trainee's historical training data.

[0077] Among them, the historical training data represents a complete data set recording the trainee's past training process, including information such as training subjects, operation records, and scoring records.

[0078] The simulation training system executes this step immediately after completing the identity verification. Specifically, the system retrieves all the historical training records of the trainee from the training record database according to the trainee ID, including the completed training subjects, detailed operation data and scoring data of each training, and structures these data to prepare for subsequent analysis.

[0079] In some embodiments, historical data acquisition can be achieved in various ways: Optionally, the system adopts a distributed storage architecture to manage training data, ensures data consistency among multiple terminals through a data synchronization mechanism, and improves data access efficiency using caching technology; Optionally, the system is based on an incremental data update strategy, synchronizes the latest training records in real time, and improves data retrieval performance through data compression and index optimization. It can be understood that other methods can also be used to achieve efficient and reliable historical data acquisition, which is not limited here.

[0080] S303. Analyze the trainee's training weak points based on the historical training data, and perform difficulty grading and priority ranking on the training subjects in the training subject library based on the training weak points.

[0081] Among them, the weak training items refer to the subjects or operation links where the trainee performs poorly during the training process; the difficulty grading is calculated based on the average completion time, average number of errors, and average score in the historical training data, and the priority is determined by the product of the difficulty level of the training subject and the lowest score of the corresponding subject in the trainee's historical training.

[0082] The simulation training system executes this step after obtaining the historical training data. Specifically, the system first analyzes the key indicators in the historical training data, including the average completion time, average number of errors, and average score, and calculates the difficulty coefficient of each subject through a mathematical model; then multiplies these difficulty coefficients by the lowest score of the trainee in the corresponding subject to obtain the subject priority index for subsequent training recommendations.

[0083] In some embodiments, the weak item analysis and subject ranking can be achieved in various ways: Optionally, the system uses machine learning algorithms to analyze the feature distribution of historical training data, identifies typical error patterns through clustering methods, and establishes a subject difficulty evaluation model; Optionally, the system is based on a multi-objective optimization algorithm, comprehensively considers factors such as training effect, time investment, and learning curve, and dynamically adjusts the subject priority. It can be understood that other methods can also be used to achieve personalized training planning, which is not limited here.

[0084] S304. Recommend training subjects to the trainee in descending order of priority, and mark the difficulty level and estimated training time on the display interface of each training subject.

[0085] Among them, the recommended training subjects refer to the training content options shown to the trainee by the system according to the priority ranking; the difficulty level is an intuitive display identifier for the difficulty of the training subject; the estimated training time represents the estimated time required to complete the training subject.

[0086] The simulation training system executes this step after completing the priority ranking. Specifically, the system constructs a training subject recommendation list according to the calculated priority order, clearly marks the difficulty level of each subject on the display interface, and gives a reasonable estimated training time based on historical data statistical analysis to help the trainee make an appropriate choice.

[0087] In some embodiments, the training recommendation can be achieved in various ways: Optionally, the system uses an adaptive recommendation algorithm, combines the trainee's learning progress and time arrangement, dynamically adjusts the recommendation strategy, and provides personalized training plan suggestions; Optionally, the system is based on knowledge graph technology, constructs the association relationship between training subjects, and generates the optimal learning path through path planning algorithms. It can be understood that other methods can also be used to achieve intelligent training recommendations, which is not limited here.

[0088] S305. Determine the target training subject according to the trainee's selection, and retrieve the corresponding standard training procedures from the training subject library.

[0089] Referring to step S101, the simulation training system will determine the standard training procedures.

[0090] S306. Determine multiple environmental factors based on the actual training scenario and construct a dynamic simulation environment.

[0091] Among them, the environmental factors include weather changes, light changes, and road condition changes, which represent external condition variables affecting the training difficulty; the dynamic simulation environment refers to the training scenario generated in real time according to the environmental factors; weather changes, light changes, and road condition changes represent a set of environmentally adjustable parameters.

[0092] The simulation training system executes this step after determining the training subject. Specifically, based on the characteristics of the current training subject, the system selects a suitable combination of environmental factors, including the change in visual range under different weather conditions, the change in light intensity at different times, the change in friction coefficient under different road conditions, etc., to construct a dynamically adjustable simulation environment system.

[0093] In some embodiments, the dynamic environment construction can be achieved in multiple ways: Optionally, the system uses a physical engine to simulate environmental changes, and realizes the real dynamic effects of weather, light, and road conditions through parametric modeling to ensure the continuity and authenticity of environmental changes; Optionally, based on the scene rendering technology, the system combines the coupling effect of environmental factors to realize the coordinated changes of multiple environmental factors and provide a more realistic training experience. It can be understood that other methods can also be used to realize the dynamic simulation of environmental factors, which are not limited here.

[0094] S307. Obtain the proficiency index of the trainee and adjust the environmental complexity of the dynamic simulation environment according to the proficiency index.

[0095] Among them, the proficiency index represents a quantitative evaluation value of the trainee's operation level; the environmental complexity refers to the difficulty coefficient and combination intensity of environmental factors in the dynamic simulation environment.

[0096] The simulation training system executes this step after constructing the dynamic simulation environment. Specifically, the system calculates the proficiency index based on the trainee's historical training data. This index comprehensively considers factors such as historical scores, completion time, and error rate, and then dynamically adjusts the parameter range and change frequency of environmental factors according to this index to achieve adaptive adjustment of training difficulty.

[0097] It should be noted that the adaptive adjustment of the dynamic environment can make use of the complexity adjustment model, which is trained based on the performance data of a large number of trainees with different proficiency levels in various environments during training. The input data includes the trainees' historical training scores, error type statistics, operation stability indicators, etc., as well as the influence coefficients of different environmental factors (weather, light, road conditions) on the training difficulty. The training standard is to match the environmental complexity with the current level of the trainees, which should be challenging to a certain extent and not exceed the trainees' ability range too much. This complexity adjustment model adopts a reinforcement learning framework, taking the trainee proficiency index as the state input, the combination of environmental factors as the action space, and the trainees' training performance as the reward signal. The model continuously adjusts the combination of environmental parameters to find the training difficulty most suitable for the current trainee level. By receiving the trainee proficiency index (calculated based on historical training data) in real time, this complexity adjustment model can output the corresponding combination of environmental factors (such as parameters like the degree of rain, light intensity, road surface conditions, etc.) to dynamically adjust the complexity of the training scenario. When the trainee performs stably at the current difficulty level, the model will gradually increase the environmental complexity to achieve a progressive increase in the training intensity.

[0098] S308. After the trainee completes the standard training subjects and selects the complex additional training option, a composite training scenario is generated based on random environmental factors.

[0099] Among them, the complex additional training option refers to the high-difficulty training option provided to the trainees; the composite training scenario refers to the comprehensive training environment formed by the combination of multiple environmental factors; the random environmental factors refer to one or more environmental variables randomly selected by the system.

[0100] The simulation training system executes this step after the trainee completes the basic training and selects the additional training. Specifically, the system randomly selects several factors from the environmental factor library and generates a composite training scenario with a certain degree of uncertainty through an algorithm to increase the challenge and practicality of the training.

[0101] Based on Figure 2 the system architecture shown above, the embodiments of this application construct a complete training subject system based on the conventional operating procedures. The trainees can select the corresponding training subjects through the training management subsystem software, including: driving along a straight road, making a U-turn in a straight line, backing up in a narrow road, backing up along a straight line, positioning for backing up, quickly backing up over a long distance, backing up along an arc, backing into a parking space in front of the garage, making a butterfly-shaped reverse into a parking space, making a direct turn, driving on an S-shaped road, driving in a misaligned alley, driving on an N-shaped road, driving on an 8-shaped road, parallel parking at an oblique position, etc. The operation steps of each training subject are established in accordance with the standard training procedures and always follow the relevant nodes of the start, operation, and end of the process for training.

[0102] The specific training process is as follows: First, the trainee needs to enter the username and password to log in to the system. After successful login, the trainee selects the corresponding training subject according to the assessment content or training requirements. At this time, the training software installed on the host of the simulation training system will retrieve the process data in the operation process library, drive the equipment simulation logic engine, generate a simulation process script to achieve simulation process interaction, and at the same time update the state machine to drive the simulation equipment model.

[0103] When the trainee operates on the virtual panel of the first display screen, the background virtual engine will capture these operation actions in real time. The engine then drives the background simulation logic engine according to the pre-set program, and displays the corresponding process principle on the second display screen interface. At the same time, the simulation engine drives the corresponding actual equipment operation animation according to the actual action logic and displays it on the third display screen interface. In this way, the system realizes the complete process from the specified practice subject to the interactive virtual practice.

[0104] After receiving the action instruction from the first display screen, the second display screen will drive the system to display the corresponding actions, processes and principles according to the pre-set instruction-action table. The animation display engine of the system is responsible for displaying the actual real-time actions, providing intuitive feedback to the trainee. At the same time, the second display screen will also drive the speaker to simulate the actual sound according to the need to enhance the realism of the training.

[0105] In some embodiments, the simulation training system will, after the trainee completes the standard training subject and selects the complex additional training option, display the road type feature options including urban roads, highways and mountain roads; based on the user option selection, determine the target road type in the road type feature options; according to the target road type, dynamic simulation environment and random environmental factors, simulate and generate a composite training scenario.

[0106] Among them, the road type feature option represents the basic feature set of different road types; the target road type refers to the training road type finally selected by the user; the composite training scenario represents the comprehensive training environment generated by combining the target road type with environmental factors; the urban road represents the urban road environment containing traffic lights, intersections and pedestrians; the highway represents the fast-passage environment with multiple lanes, ramps and service areas; the mountain road represents the complex road conditions environment containing curves, slopes and special terrains.

[0107] The simulation training system executes this step after detecting that the trainee has completed the basic training subjects and actively selects advanced training. Specifically, the system first displays three typical road types on the interface for the trainee to choose from, and each road type is accompanied by a detailed scenario description and difficulty level prompt; after the trainee confirms the selection, the system imports the characteristic parameters of the selected road type into the scenario generation engine, and at the same time combines the current dynamic environment parameters (such as weather, light, etc.) and randomly selected environmental factors (such as traffic flow, emergencies, etc.), and generates the final training scenario model through comprehensive algorithm calculation.

[0108] S309. Generate additional scoring items based on the composite training scenario.

[0109] Among them, the additional scoring items refer to the assessment content specifically set for the composite training scenario; generating based on the composite training scenario means determining the corresponding scoring criteria according to the randomly generated combination of environmental factors.

[0110] The simulation training system executes this step after generating the composite training scenario. Specifically, the system designs corresponding special scoring items according to the characteristics of the environmental factors in the current composite scenario, such as the operation stability under different weather conditions, the emergency handling ability under complex road conditions, etc., to form a complete scoring system.

[0111] In some embodiments, the generation of additional scoring items can be achieved in multiple ways: Optionally, the system uses rule engine technology to automatically generate scoring rules according to the combined characteristics of environmental factors, and ensures the rationality of scoring through weight adjustment; Optionally, the system is based on the expert system method, transforms professional experience into specific scoring criteria, and realizes the intelligent matching of scoring items. It can be understood that other ways can also be used to achieve the dynamic generation of scoring items, which are not limited here.

[0112] S310. According to the standard training procedure, display the virtual operation panel corresponding to the target training subject on the first display screen, display the working principle animation corresponding to the target training subject on the second display screen, and display the real-time simulation status corresponding to the target training subject on the third display screen.

[0113] Referring to step S102, the simulation training system will display corresponding content on each screen.

[0114] S311. Real-time collect the operation data of the trainee on the virtual operation panel, match the operation data with the operation step sequence in the standard training procedure, and generate an operation instruction sequence.

[0115] Referring to step S103, the simulation training system will generate an operation instruction sequence.

[0116] In some embodiments, when the simulation training system detects a key operation step, it converts the normal simulation mode to the intensive teaching mode; the key operation step is an operation step with an error rate exceeding a preset error threshold in the historical training data; in the intensive teaching mode, the current training process is paused, and the execution process of the current operation step is displayed in the form of an action replay in the working principle animation on the second display screen, while the standard operation demonstration and the actual operation comparison screen are synchronously displayed in a split-screen manner on the third display screen; after detecting the student's confirmation instruction to complete learning, the intensive teaching mode is converted to the normal simulation mode, and the normal training process is resumed.

[0117] Among them, the key operation step refers to the core action link that has an important impact on the training result; the intensive teaching mode refers to the state where the system starts a special teaching interaction mechanism; the preset error threshold refers to the upper limit of the acceptable error rate preset by the system; the working principle animation refers to the visual animation that shows the internal mechanism of the operation step; the action replay form refers to the display method of reproducing the operation process at a controllable speed; the standard operation demonstration refers to the reference video of the standard operation action.

[0118] The simulation training system executes this step when it identifies a key operation step during real-time monitoring. Specifically, the system first analyzes the historical training data to determine the operation links with a high error rate; when the student performs these operations, the system automatically pauses the regular training process and switches to the intensive teaching interface. In the intensive teaching interface, the second display screen shows the mechanical principle and internal structure animation involved in this operation step, and the third display screen uses a left-right split-screen method, with the standard demonstration video played on the left and the student's operation screen displayed in real time on the right for intuitive comparison. The system will wait for the student to send a confirmation signal of learning completion through a button or voice before resuming the normal training mode.

[0119] In some embodiments, the interaction of the intensive teaching mode can be achieved in various ways: Optionally, the system uses multimedia synchronization technology, first loads the three-dimensional model of the working principle, then generates animations according to the operation step sequence, and finally realizes multi-screen collaborative display through the screen synchronization algorithm; Optionally, the system is based on an interactive teaching engine, first constructs a knowledge point association network, then generates targeted teaching content, and finally realizes personalized adjustment of the teaching progress through adaptive playback control. It can be understood that other methods can also be used to achieve the intelligent interaction of the intensive teaching mode, which is not limited here.

[0120] S312. Based on the operation instruction sequence and the vehicle dynamics model, calculate the dynamic working state parameters of each vehicle component in real time.

[0121] Among them, the vehicle dynamics model refers to the mathematical model that describes the interaction relationship between each vehicle component; the dynamic working state parameters refer to various physical quantities that reflect the real-time operating state of the vehicle.

[0122] The simulation training system executes this step after generating an operation instruction sequence. Specifically, the system inputs the operation instruction sequence into a pre-established vehicle dynamics model, which contains mathematical descriptions of core components such as the engine, transmission, and braking system, and calculates the working parameters of each component in real time by solving a system of differential equations. In some embodiments, dynamic parameter calculation can be achieved in multiple ways: Optionally, the system adopts a multi-body dynamics model, considers the constraint relationships and acting forces between components, and solves the motion equations by numerical integration methods; Optionally, the system is based on a simplified model method, optimizes the calculation efficiency on the premise of ensuring calculation accuracy, and meets the requirement of real-time response. It can be understood that other methods can also be used to achieve efficient calculation of dynamic parameters, which are not limited here.

[0123] S313. According to the engine speed, vehicle speed, and road surface conditions, simulate and generate the damping characteristic curve of the clutch, and display the dynamic working state parameters and the damping characteristic curve on the third display screen.

[0124] Among them, the damping characteristic curve represents the functional relationship describing the working characteristics of the clutch; the road surface conditions refer to the road surface parameters in the current simulation environment, such as the friction coefficient, etc.

[0125] The simulation training system executes this step after calculating the dynamic working state parameters. Specifically, the system calculates the real-time damping characteristics of the clutch according to parameters such as the current engine speed, vehicle speed, and road surface conditions, generates a characteristic curve, and updates the visual display of these data in real time on the display interface.

[0126] In some embodiments, the generation of the characteristic curve can be achieved in multiple ways: Optionally, the system adopts a non-linear fitting algorithm, constructs a clutch characteristic model based on experimental data, and realizes the dynamic update of the characteristic curve through a parameter self-adaptive method; Optionally, the system is based on a physical modeling method, considers the influence of factors such as temperature and wear, and realizes more accurate characteristic prediction. It can be understood that other methods can also be used to achieve real-time calculation of the characteristic curve, which are not limited here.

[0127] S314. When it is detected that an incorrect operation causes the vehicle working state to be abnormal, trigger a warning prompt and display the stress distribution of the affected components on the second display screen.

[0128] Among them, the incorrect operation refers to improper operations that may cause vehicle damage or safety hazards; the abnormal working state refers to operating parameters that exceed the safe range; the stress distribution refers to the spatial distribution of the mechanical stress borne by the components.

[0129] The simulation training system executes this step when an abnormal state is detected. Specifically, the system monitors various working parameters in real time. When an operation beyond the safety threshold is detected, visual and audible warnings are immediately triggered. At the same time, the stress situation of the affected components is intuitively displayed on the display interface through a stress nephogram to help the trainee understand the hazards of incorrect operations.

[0130] In some embodiments, abnormal monitoring and display can be achieved in various ways: Optionally, the system adopts the finite element analysis method to calculate the stress distribution of components in real time and intuitively display the stress concentration area through heat map technology; Optionally, the system is based on a fault diagnosis model to establish the correlation between operation behaviors and fault modes to achieve early warning. It can be understood that other ways can also be adopted to realize the intelligent monitoring and visualization of abnormal states, which are not limited here.

[0131] S315. Record the actual execution time and operation deviation data of each operation step in the operation instruction sequence.

[0132] Among them, recording means saving the key data in the training process to the system database; the actual execution time refers to the actual time required to complete each operation step; the operation deviation data represents the difference between the trainee's operation and the standard requirements.

[0133] The simulation training system continuously executes this step during the training process. Specifically, the system times each operation step, records the deviations of parameters such as the position and force of the operation from the standard values, and writes these data into the training record database in real time to provide a basis for subsequent scoring and analysis.

[0134] In some embodiments, the simulation training system collects the gaze tracking data and body posture data of the trainee during the operation process; determines the safety awareness indicators including rearview mirror observation and blind spot inspection according to the gaze tracking data; determines the posture specification indicators according to the body posture data; and identifies potential hazard driving behaviors according to the safety awareness indicators and posture specification indicators.

[0135] Among them, the gaze tracking data represents a data set recording the moving trajectory of the trainee's eyes; the body posture data refers to a parameter set describing the body posture characteristics of the trainee; the safety awareness indicator represents a quantitative evaluation value of the trainee's safe observation behavior; the posture specification indicator is a scoring value measuring the standard degree of the operation posture; and the potential hazard driving behavior represents an irregular operation method that may lead to safety risks.

[0136] The simulation training system continuously executes this step throughout the training process. Specifically, the system uses a camera to collect the facial features and body movement data of the trainee in real time, and applies computer vision algorithms to extract the line-of-sight direction and body posture features. The system analyzes the line-of-sight data to evaluate the observation frequency and duration of the trainee on the rearview mirror and blind spots, and at the same time processes the body posture data to evaluate the operation norms such as sitting posture and hand position. Based on these analysis results, the system can timely identify potential dangerous driving behaviors.

[0137] In some embodiments, behavior recognition and analysis can be achieved in various ways: Optionally, the system uses deep learning technology. First, it extracts features from the collected image data, then classifies the behaviors through a trained neural network model, and finally combines an expert rule system to evaluate the behavior risk level; Optionally, the system is based on a multi-modal data fusion method. First, it aligns the line-of-sight and body posture data in time series, then constructs a behavior feature vector through feature combination, and finally applies a pattern recognition algorithm to achieve behavior analysis. It can be understood that other methods can also be used to achieve the intelligent recognition of driving behaviors, which are not limited here.

[0138] S316. Input the actual execution time and operation deviation data into a preset scoring model to obtain the final operation score.

[0139] Referring to step S105, the simulation training system will determine the final operation score.

[0140] That is, during the training process, the training management subsystem will continuously collect the relevant operation data of the trainee during the process control, and give timely prompts for incorrect operations or key steps. When the trainee completes the entire operation training subject, the system will automatically evaluate and score the trainee's performance, and conduct statistical analysis on the results. The evaluation system mainly targets the correctness of operation logic and operation behaviors, and is divided into two dimensions: qualitative evaluation and quantitative evaluation.

[0141] The qualitative evaluation mainly targets some serious operation error behaviors or clearly prohibited operation behaviors, and once they occur, they will be judged as unqualified. The quantitative evaluation targets some bad operation behaviors, deducts points cumulatively according to their severity, and the final score needs to exceed the specified score to be judged as qualified. The quantitative indicators are automatically calculated by the software according to the pre-set evaluation method, and the scores are directly displayed in the quantitative score list; while the qualitative indicators are used as additional indicators, and once such indicators appear, they will be directly judged as unqualified.

[0142] The specific assessment rule setting is based on the basic principle of the score distribution in the actual operation during training, and uses the calculation formula: Final score = Total score - (Total score / Total number of individual operation steps × Number of incorrect steps × Incorrect deduction coefficient) - (Operation time of step n / Standard time of step n) × (Total score / Individual operation steps); The system evaluates and scores by detecting the entire operation training process of the trainee, and automatically analyzes and generates reasonable guiding suggestions according to the deduction items.

[0143] S317. Generate a training report including error-prone step analysis, operation time-consuming analysis and improvement suggestions based on the final operation score and operation deviation data.

[0144] Referring to step S106, the simulation training system will generate a training report.

[0145] After the training is over, the system can display the distribution of the trainee's training scores on the third display screen. The system software uses an evaluation model established in advance according to the actual installation to comprehensively count and deeply analyze the training results of the trainees. By systematically analyzing the score distribution of the trainees, the system can accurately identify the problems and deficiencies in the training process.

[0146] The entire evaluation process can comprehensively cover all aspects of the operation process, operation principle, signal direction, operation response, state change, etc. in the actual training, and intuitively display the key elements such as the equipment connection relationship, information flow and data flow, working principle, actual demonstration, action effect, etc. in the operation process. Through this data-driven analysis method, the system can not only evaluate the current performance of the trainees, but also predict possible learning bottlenecks and formulate targeted training plans in advance.

[0147] In the embodiments of the present application, due to the adoption of the training mode of three-screen linkage to display the virtual operation panel, working principle animation and real-time simulation state, as well as innovative technologies such as personalized training plans, enhanced teaching modes, and composite training scenarios based on historical data analysis, it can provide trainees with an intuitive and systematic driving skill learning experience, effectively solve the problems of single scenario, subjective evaluation and poor effect in traditional driving training, and then achieve the training goals of being available all day long, standardized assessment and continuous improvement. It not only improves the training efficiency and quality, but also uses computer vision technology to monitor the safety awareness and driving habits of trainees in real time to help cultivate good driving behaviors.

[0148] The simulation training system in the embodiments of the present invention application will be described from the perspective of hardware processing below. Please refer to Figure 4 , which is a schematic structural diagram of an entity device of the simulation training system in the embodiments of the present application.

[0149] It should be noted that Figure 4The structure of the illustrated simulation training system is merely an example and should not impose any limitation on the functions and scope of use of the embodiments of the present invention.

[0150] As Figure 4 shown, the simulation training system includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a Read-Only Memory (ROM) 402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403, such as executing the methods described in the above embodiments. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0151] The following components are connected to the I / O interface 405: an input section 406 including an audio input device, a button switch, etc.; an output section 407 including a Liquid Crystal Display (LCD), an audio output device, an indicator light, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as required. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as required so that a computer program read from it can be installed into the storage section 408 as required.

[0152] In particular, according to the embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments of the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 409, and / or installed from the removable medium 411. When the computer program is executed by the Central Processing Unit (CPU) 401, various functions defined in the present invention are executed.

[0153] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings.

[0155] Specifically, the simulation training system of this embodiment includes a processor and a memory. A computer program is stored on the memory. When the computer program is executed by the processor, it implements the multi-interface interactive training simulation method provided in the above embodiment.

[0156] On the other hand, the present invention also provides a computer-readable storage medium. This storage medium can be included in the simulation training system described in the above embodiment; or it can exist separately and not be assembled into the simulation training system. The above storage medium carries one or more computer programs. When the above one or more computer programs are executed by a processor of a simulation training system, the simulation training system is enabled to implement the multi-interface interactive training simulation method provided in the above embodiment.

[0157] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

[0158] As used in the foregoing embodiments, depending on the context, the term "when" may be construed to mean "if" or "after" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "upon determining" or "if (the stated condition or event) is detected" may be construed to mean "if determined" or "in response to determining" or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)".

[0159] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented by a computer program instructing relevant hardware. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the foregoing method embodiments. The foregoing storage media include various media that can store program codes, such as ROM or random access memory RAM, magnetic disks, or optical discs.

Claims

1. A multi-interface interactive training simulation method, characterized in that: Applied to a simulation training system, the method comprises: Determine the target training subject according to the trainee's selection, and retrieve the standard training procedure corresponding to the target training subject from the training subject library; According to the standard training procedure, a virtual operation panel corresponding to the target training subject is displayed on the first display screen, a working principle animation corresponding to the target training subject is displayed on the second display screen, and a real-time simulation state corresponding to the target training subject is displayed on the third display screen; Collecting the trainee's operation data on the virtual operation panel in real time, matching the operation data with the operation step sequence in the standard training procedure, and generating an operation instruction sequence; Synchronously driving the working principle animation of the second display screen and the real-time simulation state of the third display screen according to the operation instruction sequence, and recording the actual execution time and operation deviation data of each operation step in the operation instruction sequence; Input the actual execution time and the operation deviation data into a preset scoring model to obtain a final score for the operation; A training report including an analysis of error-prone steps, an analysis of operation time consumption, and improvement suggestions is generated according to the final operation score and the operation deviation data.

2. The method according to claim 1, characterized in that Before the step of determining the target training subject according to the trainee's selection and retrieving the standard training procedure corresponding to the target training subject from the training subject library, the method further includes: Receive a login request from the student, and verify the identity information of the student according to a user database; After the identity verification is passed, the historical training data of the student is obtained; Analyze the training weaknesses of the trainee according to the historical training data, and perform difficulty grading and priority sorting on the training subjects in the training subject library based on the training weaknesses; the difficulty grading is calculated according to the average completion time, average number of errors and average score in the historical training data, and the priority is determined according to the product of the difficulty level of the training subject and the lowest score of the corresponding subject in the historical training of the trainee; Training subjects are recommended to the trainee in order of priority from high to low, and the difficulty level and estimated training time are marked in the display interface of each training subject.

3. The method according to claim 1, characterized in that After the step of collecting the trainee's operation data on the virtual operation panel in real time, matching the operation data with the operation step sequence in the standard training procedure, and generating an operation instruction sequence, the method further includes: When a key operation step is detected, the normal simulation mode is switched to the enhanced teaching mode; the key operation step is an operation step whose error rate in historical training data exceeds a preset error threshold; In the intensive teaching mode, the current training process is paused, and the execution process of the current operation step is displayed in the working principle animation of the second display screen in the form of action playback, and at the same time, the standard operation demonstration and the practical operation comparison screen are synchronously displayed in a split screen manner on the third display screen; After detecting that the student confirms the learning completion instruction, the intensive teaching mode is converted to the normal simulation mode to resume the normal training process.

4. The method according to claim 1, characterized in that: After the step of determining the target training subject according to the trainee's selection and retrieving the standard training procedure corresponding to the target training subject from the training subject library, the method further includes: Determine multiple environmental factors based on actual training scenarios and build a dynamic simulation environment; the environmental factors include weather changes, light changes, and road condition changes; Acquiring the proficiency index of the trainee, and adjusting the environmental complexity of the dynamic simulation environment according to the proficiency index; After the trainee completes the standard training subject and selects the complex additional training option, a composite training scenario is generated based on random environmental factors; the random environmental factors are one or more of the multiple environmental factors; Additional scoring items are generated based on the composite training scenario.

5. The method according to claim 4, characterized in that The step of generating a composite training scenario based on random environmental factors after the trainee completes the standard training subject and selects the complex additional training option specifically includes: After the student completes the standard training subjects and selects the complex additional training option, road type feature options including urban roads, highways and mountain roads are displayed; Determining a target road type in the road type feature options based on a user option selection; A composite training scenario is simulated and generated based on the target road type, the dynamic simulation environment and random environmental factors.

6. The method according to claim 1, characterized in that The step of synchronously driving the working principle animation of the second display screen and the real-time simulation state of the third display screen according to the operation instruction sequence, and recording the actual execution time and operation deviation data of each operation step in the operation instruction sequence, specifically includes: Based on the operation instruction sequence and the vehicle dynamics model, the dynamic working state parameters of each component of the vehicle are calculated in real time; Simulating and generating a damping characteristic curve of the clutch according to the engine speed, vehicle speed and road conditions, and displaying the dynamic working state parameters and the damping characteristic curve on the third display screen; When an erroneous operation is detected that causes an abnormal working state of the vehicle, a warning prompt is triggered, and the stress distribution of the affected parts is displayed on the second display screen; The actual execution time and operation deviation data of each operation step in the operation instruction sequence are recorded.

7. The method according to claim 6, characterized in that After the step of recording the actual execution time and operation deviation data of each operation step in the operation instruction sequence, the method further comprises: Collecting the trainee's eye tracking data and body posture data during the operation; determining safety awareness indicators including rearview mirror observation and blind spot checks based on the gaze tracking data; Determining posture standard indicators according to the body posture data; According to the safety awareness index and the posture standard index, driving behaviors with hidden dangers are identified.

8. A simulation training system, characterized in that: The simulation training system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the simulation training system to execute the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a simulation training system, the simulation training system is caused to execute the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product runs on a simulation training system, the simulation training system is enabled to execute the method according to any one of claims 1 to 7.

Images