Simulated driving training method and system based on actual road condition

Abstract

The embodiment of the invention provides a simulated driving training method and system based on an actual road condition, and can solve the problems that students are in a single riding state in most of time, the driving time allocated to each student is limited, and the driving efficiency is high because each student can only drive a section of journey due to the fact that the students perform driving training in turn. And the whole course route of the extraneous road test cannot be fully familiar and various driving scenes cannot be coped with. The method comprises the following steps: acquiring a simulation training operation instruction received by other driving simulation clients in the same vehicle and a timestamp associated with the simulation training operation instruction under the condition that actual driving training is carried out at a driving position; collecting video data including road condition information under the condition that actual driving training is carried out at a driving position; and after the driving training is ended, fusing the simulation training operation instruction set of each driving simulation client to the video data based on the timestamp, so that the coach evaluates the simulation driving process of other students not in the driving position.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a simulated driving training method and system based on actual road conditions. Background Art

[0002] When conducting off-road test training, usually one instructor will be assigned to each training car, and the rest will be filled with students. Students take turns to conduct actual driving training on the off-road, and the instructor will correct and guide the students during the driving process. In other words, since students take turns to conduct driving training, each student can only drive a section of the journey, resulting in students being in a single riding state most of the time. Each student is allocated limited driving time, and cannot fully familiarize themselves with the entire route of the off-road test and deal with various driving scenarios. The complexity of the sections and scenes driven by different students may vary, and the learning effect may be uneven. Summary of the invention

[0003] The embodiments of the present application provide a simulated driving training method and system based on actual road conditions, which can solve the problem that students take turns to perform driving training, and each student can only drive a section of the road, resulting in students being in a single riding state most of the time. The driving time allocated to each student is limited, and they cannot fully familiarize themselves with the entire route of the external road test and deal with various driving scenarios. The complexity of the sections and scenes driven by different students may be different, and the learning effects may be uneven.

[0004] A first aspect of an embodiment of the present application provides a simulated driving training method based on actual road conditions, comprising:

[0005] In the case of actual driving training in the driver's seat, obtaining simulation training operation instructions received by other driving simulation clients in the same vehicle and timestamps associated with the simulation training operation instructions, wherein the simulation training operation instructions are simulation driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle;

[0006] In the case of actual driving training in the driver's seat, video data including road condition information is collected;

[0007] After the driving training is finished, the simulation training operation instruction set of each driving simulation client is merged into the video data based on the timestamp, so that the coach can evaluate the simulation driving process of other students who are not in the driving seat.

[0008] Optionally, it also includes:

[0009] When conducting actual driving training in the driving seat, the collected simulation training operation instructions are matched with the ideal operation instructions, so that when the simulation training operation instructions do not match the ideal operation instructions, a problem operation prompt message is generated and displayed on the corresponding simulation client.

[0010] Optionally, the ideal operating instruction is determined based on analysis of the video data.

[0011] Optionally, also include:

[0012] In the case of actual driving training in the driver's seat, obtaining the teaching simulation operation instructions received by other teaching simulation clients in the same vehicle and the timestamp associated with the teaching simulation operation instructions, wherein the teaching simulation operation instructions are simulated driving operations performed by the instructor in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle;

[0013] When conducting actual driving training in the driving seat, the collected simulation training operation instructions are matched with the teaching simulation operation instructions, so that when the simulation training operation instructions do not match the teaching simulation operation instructions, a problem operation prompt message is generated and displayed on the corresponding simulation client.

[0014] Optionally, also include:

[0015] After the driving training is completed, the simulated driving process of other trainees who are not in the driving seat is evaluated based on the matching results of the fusion of the simulated training operation instruction sets of each driving simulation client.

[0016] Optionally, the actual driving training section performed by each trainee of the same vehicle is a part of the complete driving training section, and the complete driving training section can be obtained by splicing the actual driving training sections performed by all trainees of the same vehicle. The method further includes:

[0017] The overall driving evaluation of the student who independently completes the entire training section is predicted by combining the evaluation of the simulated driving process of the student who is not in the driving seat and the evaluation of the student in the actual driving training process.

[0018] Optionally, also include:

[0019] In the case of actual driving training in the driver's seat, collect the teaching and correction voice information of the coach in the car;

[0020] If the teaching correction voice is collected within a preset time after the actual driving operation is performed in the driving seat, the ideal operation instruction is determined based on the actual driving operation; otherwise, the ideal operation instruction is determined based on the teaching correction voice information.

[0021] A second aspect of an embodiment of the present application provides a simulated driving training device based on actual road conditions, comprising:

[0022] An acquisition unit is used to acquire, when performing actual driving training at the driver's seat, simulation training operation instructions and timestamps associated with the simulation training operation instructions received by other driving simulation clients in the same vehicle, wherein the simulation training operation instructions are simulation driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle;

[0023] A collection unit, used to collect video data including road condition information when performing actual driving training in the driving position;

[0024] The evaluation unit is used to fuse the simulation training operation instructions of each driving simulation client into the video data based on the timestamp after the driving training is completed, so that the coach can evaluate the simulation driving process of other students who are not in the driving seat.

[0025] A third aspect of an embodiment of the present application provides an electronic system, including a memory and a processor, wherein the processor is configured to implement the steps of the above-mentioned simulated driving training method based on actual road conditions when executing a computer program stored in the memory.

[0026] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned simulated driving training method based on actual road conditions are implemented.

[0027] In summary, the simulated driving training method based on actual road conditions provided in the embodiment of the present application, by performing actual driving training in the driving position, obtaining the simulated training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulated training operation instructions, the simulated training operation instructions are other students in the same vehicle with the driving position performing simulated driving operations according to the current actual road conditions of the vehicle; in the case of actual driving training in the driving position, video data including road condition information is collected; after the driving training is completed, the simulated training operation instruction set of each driving simulation client is integrated into the video data based on the timestamp, so that the coach can evaluate the simulated driving process of other students who are not in the driving position. Thus, all students participate in the operation training of actual road conditions, whether in the driving position or the non-driving position, reducing the "idle time" of students. By repeatedly practicing specific sections (such as ramp start, turning, merging, etc.) through simulation equipment, non-driving seat students can be familiar with the processing of complex scenes. The coach can analyze the performance of all students by playback, and adjust the training content in a targeted manner after finding problems. All students actively participate in the training process, forming an atmosphere of mutual learning and enhancing the overall effect of training.

[0028] Correspondingly, the simulated driving training device based on actual road conditions, the electronic system and the computer-readable storage medium provided in the embodiments of the present invention also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flowchart of a possible simulated driving training method based on actual road conditions provided in an embodiment of the present application;

[0030] Figure 2 A schematic structural block diagram of a possible simulated driving training device based on actual road conditions provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of the hardware structure of a possible driving simulation training device based on actual road conditions provided in an embodiment of the present application;

[0032] Figure 4 A schematic structural block diagram of a possible electronic system provided in an embodiment of the present application;

[0033] Figure 5 A schematic structural block diagram of a possible computer-readable storage medium provided for an embodiment of the present application. DETAILED DESCRIPTION

[0034] The embodiments of the present application provide a simulated driving training method and system based on actual road conditions, which can solve the problem that students take turns to perform driving training, and each student can only drive a section of the road, resulting in students being in a single riding state most of the time. The driving time allocated to each student is limited, and they cannot fully familiarize themselves with the entire route of the external road test and deal with various driving scenarios. The complexity of the sections and scenes driven by different students may be different, and the learning effects may be uneven.

[0035] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0036] See also Figure 1 , which is a flow chart of a simulated driving training method based on actual road conditions provided in an embodiment of the present application, and may specifically include: S110-S130.

[0037] S110, when actual driving training is performed at the driver's seat, obtain the simulation training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulation training operation instructions, wherein the simulation training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle.

[0038] S120, collecting video data including road condition information while performing actual driving training in the driver's seat.

[0039] S130, after the driving training is finished, the simulation training operation instruction set of each driving simulation client is integrated into the video data based on the timestamp, so that the coach can evaluate the simulation driving process of other trainees who are not in the driving seat.

[0040] It is understandable that the core of this method is to combine actual driving training with simulated driving training, so that students who are not in the driving position can participate in driving training through simulation equipment, and integrate their operation records with the actual road condition video data based on the timestamp synchronization mechanism. This not only improves the participation of non-driving students, but also provides coaches with a comprehensive evaluation method for all students (including driving and non-driving seats).

[0041] For example, a real driving vehicle (driving seat) can be configured for a trainee to conduct actual driving training. Real-time collection of vehicle operating status (such as speed, steering wheel angle, braking status) and road condition information (such as video, radar data). Driving simulation equipment (such as steering wheel, pedals and display screen) is configured on other seats in the vehicle. The simulation device obtains real-time road conditions through the actual sensor data of the vehicle (for example, the video stream of the camera is played on the display screen). Provide a unified time base for all devices (including the vehicle main control system, simulation equipment, and video acquisition equipment) to ensure data acquisition, operation and road condition synchronization. The driver's seat trainee operates through the real vehicle and collects its operation data (such as steering angle, throttle depth, braking strength). Real-time recording of video data of the vehicle's external road conditions, including surrounding traffic, road signs, etc. Non-driving seat trainees operate through simulation equipment, and the system records their driving instructions (steering, acceleration, braking, etc.) and timestamps. The simulated driving client collects the trainee's response to dynamic road conditions in real time (such as the handling of vehicle merging, traffic lights, front vehicle braking, etc.). After the training is completed, the operation data of the simulated driving trainee and the collected video data are fused based on the timestamp. The operation instructions of each simulated driving client correspond to the video frames of the actual driving of the vehicle according to the timestamp, forming a visual analysis record of video + operation. For example, in a simulated driving process, student A performed a "brake" operation in the simulator, and this operation will be marked in the road condition video at the time, such as the "vehicle in front suddenly decelerates" scene. The coach evaluates the driving reaction and operation rationality of each student (including actual driving and simulated driving students) by replaying the video data and operation records. The coach can view the road conditions, student operation instructions and actual results frame by frame. Compare the performance of actual driving students and simulated driving students to determine whether training needs to be strengthened for specific sections or scenes.

[0042] It is understandable that each student's operation must accurately match the time point when the vehicle drives to the corresponding road section. The timestamp mechanism ensures the authenticity and consistency of the simulation training and avoids training distortion caused by time offset. Data fusion integrates video and operation instructions to form a traceable learning resource. In this way, the coach can conduct an in-depth analysis of the student's performance. For example, student B did not avoid the vehicle that suddenly changed lanes in time during the simulated driving. After discovering this problem through playback, the corresponding training can be focused on. Non-driving trainees actively participate in the training through simulated driving equipment to avoid the problem of distraction caused by "passive riding" in the traditional mode. For example, on a complex road section (such as a roundabout or a highway entrance and exit), student C uses the simulation equipment to familiarize himself with the driving strategy in advance, which helps his performance in actual driving. Even if the non-driving trainee does not actually operate the vehicle, the training of the simulation equipment can still improve the perception of road conditions and driving judgment ability.

[0043] In summary, the simulated driving training method based on actual road conditions provided in the above embodiment obtains the simulated training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulated training operation instructions when performing actual driving training in the driving position, and the simulated training operation instructions are simulated driving operations performed by other students in the same vehicle with the driving position according to the current actual road conditions of the vehicle; when performing actual driving training in the driving position, video data including road condition information is collected; after the driving training is completed, the simulated training operation instruction set of each driving simulation client is integrated into the video data based on the timestamp, so that the coach can evaluate the simulated driving process of other students who are not in the driving position. As a result, all students participate in the operation training of actual road conditions, whether in the driving position or the non-driving position, reducing the "idle time" of students. By repeatedly practicing specific sections (such as ramp start, turning, merging, etc.) through simulation equipment, non-driving seat students can be familiar with the processing of complex scenes. The coach can analyze the performance of all students by playback, and adjust the training content in a targeted manner after finding problems. All students actively participate in the training process, forming an atmosphere of mutual learning and enhancing the overall effect of training.

[0044] In one embodiment, it also includes:

[0045] When conducting actual driving training in the driving seat, the collected simulation training operation instructions are matched with the ideal operation instructions, so that when the simulation training operation instructions do not match the ideal operation instructions, a problem operation prompt message is generated and displayed on the corresponding simulation client.

[0046] It is understandable that in order to further improve the effect of simulated driving training, an instant feedback mechanism based on ideal operation instruction matching is added. By comparing the trainee's operation on the simulated driving device with the preset ideal operation instruction, a problem operation prompt message is generated when a deviation is found, and it is displayed on the corresponding simulation client to guide the trainee to improve the operation in real time.

[0047] Exemplarily, the trainee completes real-time operations (such as steering, braking, acceleration, etc.) through the simulated driving client, and the system records each step of the operation and its timestamp. For example, the current simulated trainee operation is "turn left 15°", and the operation time is T1. The ideal operation instruction is determined by the following two methods: Preset rules: define ideal operations for standard driving scenarios (such as straight roads, turns, and lane changes). For example: when the current vehicle speed is 50km / h, it is necessary to turn left 100 meters ahead, and the ideal instruction is "turn left at an angle of 30° 20 meters in advance". Or real-time calculation: based on the vehicle status and road conditions (such as the distance of the vehicle in front, the radius of the curve), dynamically generate ideal operations. For example, the vehicle is traveling at 60km / h, and the distance from the front vehicle is 50 meters. The ideal operation generated by the system is "slow down to 40km / h". The operation instructions collected by the simulated driving client are matched with the ideal operation instructions in real time to determine whether they meet the specifications, for example, the angle deviation (such as the steering angle) is less than the set threshold, the speed change is reasonable, and the brake advance meets the requirements. If the simulated operation instruction does not match the ideal instruction, the system generates a problem prompt message and displays it on the corresponding simulated client. The prompt information includes: describing the specific problem of the operation, such as "the steering angle is too small" and "the brakes are too late". It can also be a suggestion for improvement, such as "slow down to 40km / h in advance". The prompt message is fed back to the trainee in real time through text, icons or voice to help them correct the problem operation immediately. For example: "Currently understeering, please increase the steering angle to 30°." or "Braking too late, please start slowing down 50 meters ahead." As a result, the trainee receives instant problem prompts during the training process, which helps to quickly understand the cause of the error and correct the operation without having to wait for unified feedback after the training. The ideal operation instructions are set based on actual driving specifications. By comparing with the trainee's operation, the trainee gradually internalizes the correct driving habits. Real-time prompts enhance the sense of substitution of non-driving trainees in simulation training, making them more focused on the operation and closer to the actual driving experience. Problem prompts are automatically generated by the system, and the instructor does not need to manually point out the mistakes of each trainee, so that more energy can be devoted to overall guidance.

[0048] In one embodiment, the ideal operating instructions are determined based on analysis of the video data.

[0049] Exemplarily, the ideal operating instructions are no longer fixed preset rules or simply calculated through vehicle status, but are generated through dynamic analysis based on the collected video data. This method generates ideal operating instructions that are more in line with the actual scene through a detailed understanding of the actual road conditions, thereby improving the pertinence and practicality of the training. As a result, the ideal instructions based on video data analysis can reflect the current road conditions in real time, dynamically adapt to different driving scenarios, and are closer to reality than fixed rules. Based on the results of video analysis, the system generates ideal operations for complex scenarios (such as merging, roundabouts, rainy days, etc.) to help students better deal with special situations. During the operation, students receive video analysis prompts based on real-time road conditions, which can better adapt to the dynamically changing driving environment. The ideal operating instructions generated by video analysis serve as an evaluation benchmark to help coaches quickly discover students' weaknesses and provide specific suggestions for improvement.

[0050] In one embodiment, it also includes:

[0051] In the case of actual driving training in the driver's seat, obtaining the teaching simulation operation instructions received by other teaching simulation clients in the same vehicle and the timestamp associated with the teaching simulation operation instructions, wherein the teaching simulation operation instructions are simulated driving operations performed by the instructor in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle;

[0052] When conducting actual driving training in the driving seat, the collected simulation training operation instructions are matched with the teaching simulation operation instructions, so that when the simulation training operation instructions do not match the teaching simulation operation instructions, a problem operation prompt message is generated and displayed on the corresponding simulation client.

[0053] It is understandable that the coach participates in the driving operation simulation through the teaching simulation client and gives the most reasonable driving operation (i.e. teaching simulation operation instruction) based on the real-time road conditions. The system matches the student's simulated driving operation instruction with the coach's operation instruction in real time, and generates a problem operation prompt message for mismatches, guiding the student to adjust the operation immediately. This method provides students with more targeted and authoritative operation references by introducing the coach's guidance behavior.

[0054] Exemplarily, the coach operates through the teaching simulation client (such as steering wheel, pedals, steering device, etc.) to simulate ideal driving behavior. The system records the coach's operating instructions (such as steering angle, acceleration or braking force) and their timestamps to ensure synchronization. The student operates through the simulated driving client, and the system records its operating instructions and timestamps in real time. The system binds the coach's and the student's operating instructions to the same timeline to ensure that the instructions of the two can be matched and compared based on the same actual road conditions. The matching rules can be set according to specific driving tasks, for example: Steering matching: The error between the student's steering angle and the coach's instruction angle is lower than the set threshold (such as ±5°). Speed ​​matching: The student's acceleration or deceleration operation and the coach's operation error are within a reasonable range (such as ±5km / h). Braking matching: The difference between the student's braking force and the coach's braking force is lower than the set value. If the student's operation does not match the coach's instructions, the system records the type, degree and time point of the deviation. When the match fails, the system generates prompt information based on the deviation, which may include: describing the student's specific operation problem, such as "oversteering", "insufficient braking" and "unreasonable speed control". Or provide operation improvement methods, such as "Please reduce the steering angle to 20°" or "Lightly apply the brakes in advance". The prompt information is displayed in real time through the simulation client, and the prompt content can be: text prompt: "Current oversteering, please adjust the steering angle to 15°."; ​​Graphic prompt: Dynamically display the deviation direction and suggestions in the form of icons; Voice prompt: "Accelerating too fast, please slow down." The coach can provide targeted guidance to the trainee by comparing the playback teaching simulation operation instructions with the trainee's simulation operation instructions. For example, in a complex lane-changing operation, the trainee turned too early and did not control the speed of the vehicle. Through playback, the coach can point out the problem and provide further operation guidance. The playback content may include: the coach's ideal operation and the trainee's actual operation curve or the actual road condition video and the corresponding operation time. Therefore, by introducing the coach's operation as the ideal operation instruction, the trainee can obtain more targeted and credible guidance. For example, in complex sections (such as lane merging and roundabouts), the coach's real-time operation can help the trainee understand the best response of professional drivers. Instant question prompts help trainees quickly understand the problem and adjust the operation without having to wait for centralized feedback after the training. For example, if a trainee fails to slow down in time before a zebra crossing, the system prompts "Please lightly apply the brakes in advance", and the trainee can make adjustments immediately. The instructor's operation can respond to complex road conditions (such as sudden braking of the vehicle in front, traffic congestion) in real time and generate dynamic ideal instructions that adapt to a variety of scenarios. For example, when entering a high-speed ramp, the instructor's operation can provide the trainee with a direct reference for steering angle and speed control. By comparing their own operations with the instructor's operations, the trainee can gradually master how to make more reasonable decisions in different driving scenarios.

[0055] In one embodiment, it also includes:

[0056] After the driving training is completed, the simulated driving process of other trainees who are not in the driving seat is evaluated based on the matching results of the fusion of the simulated training operation instruction sets of each driving simulation client.

[0057] It is understandable that by matching and analyzing the operation instructions of the trainee in the simulated driving client who is not in the driving position with the ideal operation instructions (or the coach's teaching simulation instructions), a quantitative evaluation result based on the operation deviation is generated. This method can provide the trainee with comprehensive feedback on his simulated driving operation after the training, helping the trainee understand his performance and develop a personalized improvement plan.

[0058] For example, students can clearly understand their specific problems in simulated driving and get targeted suggestions for improvement. For example, a student frequently deviates from the ideal angle in turns. Through the evaluation report, it is found that the main problem is too fast steering, and it is recommended that he do more slow turning training. Students who are not in the driving seat can also get detailed evaluations similar to those of students in the actual driving seat to avoid affecting the learning effect due to "passive observation". For example, through the comprehensive scoring system, the performance of each student is quantified to ensure the fairness of the evaluation. The coach can quickly understand the weak links of the students based on the evaluation report of the students and optimize the teaching strategy. For example, if most students score low when merging at high speed, the coach can focus on explaining the merging skills in subsequent training. The operation of the simulated driving client is no longer just an auxiliary training, but has become an important basis for evaluation, prompting students to participate more actively.

[0059] According to some embodiments, the actual driving training section performed by each trainee of the same vehicle is a part of the complete driving training section, and the complete driving training section can be obtained by splicing the actual driving training sections performed by all trainees of the same vehicle, and the method further includes:

[0060] The overall driving evaluation of the student who independently completes the entire training section is predicted by combining the evaluation of the simulated driving process of the student who is not in the driving seat and the evaluation of the student in the actual driving training process.

[0061] It is understandable that each student only conducts actual driving training in a part of the complete training section, and the complete section is composed of the actual driving sections of all students in the same vehicle. In order to predict the comprehensive driving ability of each student, the following two evaluation indicators are combined: the student's simulated driving process evaluation: the score and evaluation generated based on the simulated operation data when the student is not in the driving seat. The student's actual driving training process evaluation: the score and evaluation generated based on the real operation data when the student is in the actual driving seat. By integrating the results of these two parts and combining them with the road condition information of the complete section, the overall driving performance of the student when completing the complete training section independently is predicted.

[0062] Exemplarily, the operation data of the trainee through the simulated client when not in the driving position may include: steering angle, braking force, acceleration depth and other operation instructions. The timestamp corresponding to the operation. The deviation analysis results automatically generated by the system. The real operation data of the trainee in the actual driving position may include: operation accuracy, reaction timeliness, and full-course coherence. Road condition information during driving (such as lane curvature, dynamic obstacles, etc.). The actual driving sections of each trainee are spliced ​​in sequence on the timeline to generate a complete driving training section. All related road condition information and operation requirements are retained when the sections are spliced. The trainee's simulated driving operation is gradually matched with the ideal operation instructions to generate the following key indicators: Operation accuracy score: the proportion of the trainee's operation with a small deviation from the ideal operation. Reaction timeliness score: the trainee's response time to complex road conditions (such as the brakes of the front vehicle and traffic lights). Scenario adaptability score: the trainee's operation performance in different scenarios (such as high speeds and curves). For example, the simulation driving evaluation score: 85 points (good). The student's performance in the actual driving process is comprehensively evaluated, and the following key indicators are calculated: Operational fluency: the smoothness of braking, acceleration and steering. Scenario expression: the quality of operation for specific road conditions (such as merging and turning). Safety score: whether there are dangerous operations (such as sudden braking and failure to use turn signals). For example, the actual driving evaluation score is 78 points (good). The student's simulated driving evaluation and actual driving evaluation are combined to predict his driving performance on the complete training section. The weight model is used to balance the contribution of simulated driving and actual driving evaluation to the comprehensive evaluation. The weight value can be adjusted according to the training purpose. Comprehensive evaluation score = α × simulated driving score + β × actual driving score, where α and β are weight coefficients (such as 0.4 and 0.6). The comprehensive score can be corrected according to the complexity of different sections in the complete section. For example, the score of the highway section has a high weight. The weight of the simple straight section is low. The student's driving performance level of the complete section can be divided according to the comprehensive score. As a result, the student only completes the actual driving training of part of the section, but combined with the simulated driving process data, his comprehensive performance on the complete section can be effectively predicted. Comprehensive evaluation combines the students' simulation and actual performance to provide more targeted and comprehensive feedback. Through prediction and reporting, coaches can more effectively plan the focus of the next stage of training and improve teaching efficiency. Students can clarify their own strengths and weaknesses through evaluation reports and increase their confidence in driving on the entire road section.

[0063] According to some embodiments, further comprising:

[0064] In the case of actual driving training in the driver's seat, collect the teaching and correction voice information of the coach in the car;

[0065] If the teaching correction voice is collected within a preset time after the actual driving operation is performed in the driving seat, the ideal operation instruction is determined based on the actual driving operation; otherwise, the ideal operation instruction is determined based on the teaching correction voice information.

[0066] It is understandable that in driving training, the coach's instant voice correction provides authoritative guidance and can directly reflect the ideal operation in a specific scenario. This embodiment dynamically generates ideal operation instructions by collecting the coach's teaching correction voice information in the car and combining it with the trainee's actual operation, ensuring that the ideal instructions are highly consistent with the scenario and the coach's teaching strategy. If the coach's voice information is not collected within the set time after the actual driving, the system automatically generates the ideal operation instructions based on the actual driving operation.

[0067] For example, during driving training, the coach's voice information is collected through the vehicle-mounted microphone.

[0068] The voice information includes: correction content, such as "brake in advance" and "slow down and turn right". The timestamp corresponds to the specific time point of the voice correction, which is synchronized with the student's operation and road conditions. Voice collection supports noise filtering and keyword recognition. Collect the student's operation data in the driving position (such as steering angle, braking force, speed, etc.) and its timestamp. Collect road condition video through the on-board camera, including dynamic traffic scenes (such as traffic lights, pedestrians, and the status of the vehicle in front). If the coach's voice information is collected within a preset time (such as 5 seconds) after the student actually drives, the ideal operation instruction is generated based on the coach's correction content. Perform semantic analysis on the voice information and extract keywords (such as "brake", "slow down", "turn left"). Match the driving scene according to the voice content, and generate specific ideal operations based on the road conditions and vehicle status. If the coach's voice correction is not collected, the ideal operation instruction is generated based on the actual driving operation and road condition data. Generate the ideal operation based on the dynamic state of the vehicle (such as speed and direction) combined with standard driving specifications. Integrate the coach's voice with the student's operation and road conditions to form an ideal operation instruction. If the coach's voice correction content matches the student's operation, it verifies whether the student's operation achieves the ideal effect. If the coach's voice correction content does not match the student's operation, a question prompt is generated and the student's deviation is recorded. As a result, the coach's voice correction information directly participates in the generation of ideal instructions, ensuring consistency with the training goals and providing guidance that is highly consistent with the scenario. When voice information is not collected, the system can automatically generate ideal operation instructions to ensure that the student's operation evaluation is not disturbed. By analyzing the match between the coach's corrected voice and the student's operation, the coach can quickly understand the student's weak links and adjust the teaching content accordingly. Through voice correction and system prompts, students can instantly understand the operation deviation and make adjustments.

[0069] See also Figure 2, an embodiment of the simulated driving training device based on actual road conditions in the embodiments of the present application may include:

[0070] The acquisition unit 201 is used to acquire, when the actual driving training is performed at the driver's seat, the simulation training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulation training operation instructions, wherein the simulation training operation instructions are the simulation driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle;

[0071] A collection unit 202 is used to collect video data including road condition information when performing actual driving training in the driving position;

[0072] The evaluation unit 203 is used to merge the simulation training operation instructions of each driving simulation client into the video data based on the timestamp after the driving training is completed, so that the coach can evaluate the simulation driving process of other students who are not in the driving seat.

[0073] In summary, the simulated driving training device based on actual road conditions provided in the above embodiment obtains the simulated training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulated training operation instructions when performing actual driving training in the driving position, and the simulated training operation instructions are simulated driving operations performed by other students in the same vehicle with the driving position according to the current actual road conditions of the vehicle; when performing actual driving training in the driving position, video data including road condition information is collected; after the driving training is completed, the simulated training operation instruction set of each driving simulation client is integrated into the video data based on the timestamp, so that the coach can evaluate the simulated driving process of other students who are not in the driving position. As a result, all students participate in the operation training of actual road conditions, whether in the driving position or the non-driving position, reducing the "idle time" of students. By repeatedly practicing specific sections (such as ramp start, turning, merging, etc.) through the simulation device, non-driving seat students can be familiar with the processing of complex scenes. The coach can analyze the performance of all students by playback, and adjust the training content in a targeted manner after finding problems. All students actively participate in the training process, forming an atmosphere of mutual learning and enhancing the overall effect of training.

[0074] above Figure 2 The simulated driving training device based on actual road conditions in the embodiment of the present application is described from the perspective of modular functional entities. The simulated driving training device based on actual road conditions in the embodiment of the present application is described in detail from the perspective of hardware processing. Please refer to Figure 3 , an embodiment of a simulated driving training device 300 based on actual road conditions in an embodiment of the present application includes:

[0075] An input device 301, an output device 302, a processor 303 and a memory 304, wherein the number of the processor 303 can be one or more. Figure 3 In some embodiments of the present application, the input device 301, the output device 302, the processor 303 and the memory 304 may be connected via a bus or other means, wherein: Figure 3 The example of connecting through bus is taken in the following.

[0076] Wherein, by calling the operation instruction stored in the memory 304, the processor 303 is used to perform the following steps:

[0077] In the case of actual driving training in the driver's seat, obtaining simulation training operation instructions received by other driving simulation clients in the same vehicle and timestamps associated with the simulation training operation instructions, wherein the simulation training operation instructions are simulation driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle;

[0078] In the case of actual driving training in the driver's seat, video data including road condition information is collected;

[0079] After the driving training is finished, the simulation training operation instruction set of each driving simulation client is merged into the video data based on the timestamp, so that the coach can evaluate the simulation driving process of other students who are not in the driving seat.

[0080] By calling the operation instructions stored in the memory 304, the processor 303 is also used to execute Figure 1 Any method in the corresponding embodiment.

[0081] See also Figure 4 , Figure 4 A schematic diagram of an electronic system according to an embodiment of the present application.

[0082] like Figure 4 As shown, an embodiment of the present application provides an electronic system, including a memory 410, a processor 420, and a computer program 411 stored in the memory 420 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented:

[0083] In the case of actual driving training in the driver's seat, obtaining simulation training operation instructions received by other driving simulation clients in the same vehicle and timestamps associated with the simulation training operation instructions, wherein the simulation training operation instructions are simulation driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle;

[0084] In the case of actual driving training in the driver's seat, video data including road condition information is collected;

[0085] After the driving training is finished, the simulation training operation instruction set of each driving simulation client is merged into the video data based on the timestamp, so that the coach can evaluate the simulation driving process of other students who are not in the driving seat.

[0086] In the specific implementation process, when the processor 420 executes the computer program 411, it can achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0087] Since the electronic system introduced in this embodiment is a device used to implement a simulated driving training device based on actual road conditions in the embodiment of the present application, based on the method introduced in the embodiment of the present application, the technical personnel in this field can understand the specific implementation method of the electronic system of this embodiment and its various variations. Therefore, how the electronic system implements the method in the embodiment of the present application is not introduced in detail here. As long as the equipment used by the technical personnel in this field to implement the method in the embodiment of the present application is within the scope of protection of this application.

[0088] See also Figure 5 , Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present application.

[0089] like Figure 5 As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, the following steps are implemented:

[0090] In the case of actual driving training in the driver's seat, obtaining simulation training operation instructions received by other driving simulation clients in the same vehicle and timestamps associated with the simulation training operation instructions, wherein the simulation training operation instructions are simulation driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle;

[0091] In the case of actual driving training in the driver's seat, video data including road condition information is collected;

[0092] After the driving training is finished, the simulation training operation instruction set of each driving simulation client is merged into the video data based on the timestamp, so that the coach can evaluate the simulation driving process of other students who are not in the driving seat.

[0093] In the specific implementation process, when the computer program 511 is executed by the processor, it can achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0094] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0096] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] The present application also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device is caused to execute the following Figure 1The process in the simulated driving training method based on actual road conditions in the corresponding embodiment.

[0100] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0101] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0102] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0105] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0106] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A simulated driving training method based on actual road conditions, characterized in that: include: In the case of actual driving training in the driver's seat, obtaining simulation training operation instructions received by other driving simulation clients in the same vehicle and timestamps associated with the simulation training operation instructions, wherein the simulation training operation instructions are simulation driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle; In the case of actual driving training in the driver's seat, video data including road condition information is collected; After the driving training is finished, the simulation training operation instruction set of each driving simulation client is merged into the video data based on the timestamp, so that the coach can evaluate the simulation driving process of other students who are not in the driving seat.

2. The method according to claim 1, characterized in that Also includes: When conducting actual driving training in the driving seat, the collected simulation training operation instructions are matched with the ideal operation instructions, so that when the simulation training operation instructions do not match the ideal operation instructions, a problem operation prompt message is generated and displayed on the corresponding simulation client.

3. The method according to claim 2, characterized in that The ideal operating instructions are determined based on the analysis of the video data.

4. The method according to claim 1, characterized in that: Also includes: In the case of actual driving training in the driver's seat, obtaining the teaching simulation operation instructions received by other teaching simulation clients in the same vehicle and the timestamp associated with the teaching simulation operation instructions, wherein the teaching simulation operation instructions are simulated driving operations performed by the instructor in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle; When conducting actual driving training in the driving seat, the collected simulation training operation instructions are matched with the teaching simulation operation instructions, so that when the simulation training operation instructions do not match the teaching simulation operation instructions, a problem operation prompt message is generated and displayed on the corresponding simulation client.

5. The method according to any one of claims 2 to 4, characterized in that Also includes: After the driving training is completed, the simulated driving process of other trainees who are not in the driving seat is evaluated based on the matching results of the fusion of the simulated training operation instruction sets of each driving simulation client.

6. The method according to claim 5, characterized in that The actual driving training section for each student of the same vehicle is a part of the complete driving training section. The complete driving training section can be obtained by splicing the actual driving training sections for all students of the same vehicle. The method further includes: The overall driving evaluation of the student who independently completes the entire training section is predicted by combining the evaluation of the simulated driving process of the student who is not in the driving seat and the evaluation of the student in the actual driving training process.

7. The method according to claim 2, characterized in that Also includes: In the case of actual driving training in the driver's seat, collect the teaching and correction voice information of the coach in the car; If the teaching correction voice is collected within a preset time after the actual driving operation is performed in the driving seat, the ideal operation instruction is determined based on the actual driving operation; otherwise, the ideal operation instruction is determined based on the teaching correction voice information.

8. A driving simulation training device based on actual road conditions, characterized in that: include: An acquisition unit is used to acquire, when performing actual driving training at the driver's seat, simulation training operation instructions and timestamps associated with the simulation training operation instructions received by other driving simulation clients in the same vehicle, wherein the simulation training operation instructions are simulation driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle; A collection unit, used to collect video data including road condition information when performing actual driving training in the driving position; The evaluation unit is used to fuse the simulation training operation instructions of each driving simulation client into the video data based on the timestamp after the driving training is completed, so that the coach can evaluate the simulation driving process of other students who are not in the driving seat.

9. An electronic system, comprising a memory and a processor, characterized in that: The processor is used to implement the steps of the simulated driving training method based on actual road conditions as described in any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the simulated driving training method based on actual road conditions are implemented as claimed in any one of claims 1 to 7.

Images