Simulation driving training method and system based on actual road conditions

Abstract

The embodiment of the application provides a kind of simulation driving training method and system based on actual road condition, can solve since trainee rotation is carried out driving training, each trainee can only drive one-way journey, leading to most of the time trainee is in single ride state, the driving time allocated to each trainee is limited, cannot be fully familiar with the whole route of external road test and deal with a variety of driving scene problem.The method comprises: in the case of actual driving training in driving position, the simulation training operation instruction received by other driving simulation client in the same vehicle and the timestamp associated with the simulation training operation instruction are obtained;In the case of actual driving training in driving position, video data including road condition information is collected;After ending driving training, the simulation training operation instruction set of each driving simulation client is fused into the video data based on the timestamp, so that the simulation driving process of other trainees not in driving position is evaluated by the coach.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a simulated driving training method and system based on actual road conditions. Background Technology

[0002] During road test training, each training vehicle typically has one instructor, while the rest of the seats are occupied by students. Students take turns practicing driving on the actual road, with the instructor providing corrections and guidance during the process. In other words, because students take turns, each student can only drive a portion of the route, resulting in them spending most of their time in a single passenger position. The limited driving time allocated to each student prevents them from fully familiarizing themselves with the entire road test route and handling various driving scenarios. Furthermore, the complexity of the routes and scenarios experienced by different students may vary, leading to inconsistent learning outcomes. Summary of the Invention

[0003] This application provides a simulated driving training method and system based on actual road conditions, which can solve the problems that, due to students taking turns to conduct driving training, each student can only drive a section of the road, resulting in students being in a single passenger state for most of the time, and each student having limited driving time, making it impossible to fully familiarize themselves with the entire route of the external road test and cope with various driving scenarios. The complexity of the road sections and scenarios driven by different students may vary, and the learning effect may be uneven.

[0004] The first aspect of this application provides a simulated driving training method based on actual road conditions, including:

[0005] In the case of actual driving training in the driver's seat, the simulation training operation instructions and the timestamp associated with the simulation training operation instructions received by other driving simulation clients in the same vehicle are obtained. The simulation training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle.

[0006] Collect video data, including road condition information, while conducting actual driving training in the driver's seat;

[0007] After the driving training ends, the set of simulated training operation instructions from each driving simulation client is merged into the video data based on timestamps, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

[0008] Optionally, it also includes:

[0009] When conducting actual driving training in the driver's seat, the collected simulated training operation commands are matched with ideal operation commands. If the simulated training operation commands and ideal operation commands do not match, a problem operation prompt message is generated and displayed on the corresponding simulation client.

[0010] Optionally, the ideal operating instructions are determined based on the video data analysis.

[0011] Optional, also includes:

[0012] In the case of actual driving training in the driver's seat, the teaching simulation operation instructions and the timestamp associated with the teaching simulation operation instructions received by other teaching simulation clients in the same vehicle are obtained. The teaching simulation operation instructions are simulated driving operations performed by the instructor in the same vehicle as the driver's seat based on the current actual road conditions of the vehicle.

[0013] In the case of actual driving training in the driver's seat, the collected simulated training operation instructions are matched with the teaching simulated operation instructions. If the simulated training operation instructions and the teaching simulated operation instructions do not match, a problem operation prompt message is generated and displayed on the corresponding simulation client.

[0014] Optional, also includes:

[0015] After the driving training is completed, the simulated driving process of other students who are not in the driver's seat is evaluated based on the matching results of the fusion of the simulated training operation command sets of each driving simulation client.

[0016] Optionally, the actual driving training route for each student in the same vehicle is part of the complete driving training route. The complete driving training route can be obtained by splicing together the actual driving training routes for all students in the same vehicle. The method further includes:

[0017] By combining the evaluation of the student's simulated driving process when not in the driver's seat with the evaluation of the student during the actual driving training process, the overall driving evaluation of the student who independently completes the entire training route is predicted.

[0018] Optional, also includes:

[0019] During actual driving training in the driver's seat, the instructor's voice information for teaching corrections is collected from inside the vehicle.

[0020] If no corrective teaching voice is collected within a preset time after actual driving operation in the driver's seat, the ideal operating instruction is determined based on the actual driving operation; otherwise, the ideal operating instruction is determined based on the corrective teaching voice information.

[0021] A second aspect of this application provides a simulated driving training device based on actual road conditions, comprising:

[0022] The acquisition unit is used to acquire, when actual driving training is being conducted in the driver's seat, the simulated training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulated training operation instructions, wherein the simulated training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat based on the current actual road conditions of the vehicle.

[0023] The acquisition unit is used to collect video data, including road condition information, during actual driving training in the driver's seat.

[0024] An evaluation unit is used to integrate the simulated training operation instructions of each driving simulation client into the video data based on timestamps after the driving training ends, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

[0025] A third aspect of this application provides an electronic system including a memory and a processor, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the above-described simulated driving training method based on actual road conditions.

[0026] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described simulated driving training method based on actual road conditions.

[0027] In summary, the simulated driving training method based on actual road conditions provided in this application obtains simulated training operation instructions and associated timestamps received by other driving simulation clients within the same vehicle during actual driving training in the driver's seat. These simulated training operation instructions are simulated driving operations performed by other learners within the same vehicle based on the current actual road conditions. During actual driving training in the driver's seat, video data including road condition information is collected. After the driving training ends, the simulated training operation instruction sets of each driving simulation client are merged into the video data based on the timestamps, allowing the instructor to evaluate the simulated driving process of other learners not in the driver's seat. Thus, all learners participate in operational training under actual road conditions, whether in the driver's seat or not, reducing "idle time." By repeatedly practicing specific road sections (such as hill starts, turns, lane changes, etc.) through the simulation equipment, learners not in the driver's seat can become familiar with handling complex scenarios. The instructor can analyze the performance of all learners through playback, identify problems, and adjust the training content accordingly. All learners actively participate in the training process, creating a mutual learning atmosphere and enhancing the overall training effect.

[0028] Correspondingly, the simulated driving training device, electronic system, and computer-readable storage medium based on actual road conditions provided in the embodiments of the present invention also have the above-mentioned technical effects. Attached Figure Description

[0029] Figure 1 A flowchart illustrating a possible simulated driving training method based on actual road conditions, provided for an embodiment of this application;

[0030] Figure 2 A schematic structural block diagram of a possible simulated driving training device based on actual road conditions, provided for an embodiment of this application;

[0031] Figure 3 A schematic diagram of the hardware structure of a possible simulated driving training device based on actual road conditions, provided for an embodiment of this application;

[0032] Figure 4 A schematic structural block diagram of a possible electronic system provided for embodiments of this application;

[0033] Figure 5 This is a schematic structural block diagram of a possible computer-readable storage medium provided for embodiments of this application. Detailed Implementation

[0034] This application provides a simulated driving training method and system based on actual road conditions, which can solve the problems that, due to students taking turns to conduct driving training, each student can only drive a section of the road, resulting in students being in a single passenger state for most of the time, and each student having limited driving time, making it impossible to fully familiarize themselves with the entire route of the external road test and cope with various driving scenarios. The complexity of the road sections and scenarios driven by different students may vary, and the learning effect may be uneven.

[0035] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions of the embodiments of this application will now be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0036] Please see Figure 1 The flowchart of a simulated driving training method based on actual road conditions provided in this application embodiment may specifically include: S110-S130.

[0037] S110, when conducting actual driving training in the driver's seat, obtain the simulation training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulation training operation instructions. The simulation training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver based on the current actual road conditions of the vehicle.

[0038] The S120 collects video data, including road condition information, during actual driving training in the driver's seat.

[0039] S130, after the driving training ends, the set of simulated training operation instructions for each driving simulation client is merged into the video data based on the timestamp, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

[0040] Understandably, the core of this method lies in combining actual driving training with simulated driving training. This allows non-driving students to participate in driving training through simulation equipment, and their operational records are integrated with real-world road condition video data based on a timestamp synchronization mechanism. This not only increases the participation of non-driving students but also provides instructors with a comprehensive evaluation tool for all students (including those in and out of the driver's seat).

[0041] For example, a real driving vehicle (driver's seat) can be configured for a student to conduct actual driving training. Real-time data is collected on the vehicle's operating status (e.g., speed, steering wheel angle, braking status) and road condition information (e.g., video, radar data). Driving simulation devices (e.g., steering wheel, pedals, and displays) are configured in other seats within the vehicle. The simulation devices acquire real-time road conditions through the vehicle's actual sensor data (e.g., video streams from cameras played on the display). A unified time reference is provided for all devices (including the vehicle's main control system, simulation devices, and video acquisition devices) to ensure synchronization of data acquisition, operation, and road conditions. The student in the driver's seat operates the real vehicle, and their operational data (e.g., steering angle, throttle depth, braking intensity) is collected. Real-time video data of external road conditions, including surrounding traffic and road signs, is recorded. The non-driver's seat student operates the simulation devices, and the system records their driving commands (steering, acceleration, braking, etc.) and timestamps. The driving simulator client collects the student's responses to dynamic road conditions in real time (e.g., handling of traffic merging, traffic lights, and braking of the vehicle in front). After training, the simulated driving student's operational data and the collected video data are fused based on timestamps. Each simulated driving client's operational commands are matched with video frames of the actual vehicle driving according to timestamps, forming a visual analysis record of video and operation. For example, during a simulated driving session, student A performs a "brake" operation in the simulator; this operation is marked in the road condition video at that time, such as a "vehicle ahead suddenly decelerates" scenario. The instructor evaluates the driving reactions and operational rationality of each student (including both actual and simulated driving students) by replaying the video data and operation records. The instructor can view the road conditions, student operational commands, and actual results frame by frame. By comparing the performance of actual and simulated driving students, the instructor can determine whether specific road sections or scenarios require additional training.

[0042] Understandably, each student's actions must accurately match the time the vehicle reaches the corresponding road segment. The timestamp mechanism ensures the realism and consistency of the simulated training, avoiding training distortion due to time shifts. Data fusion integrates video and operational instructions, creating a traceable learning resource. In this way, instructors can conduct in-depth analysis of student performance. For example, if student B fails to avoid a vehicle suddenly changing lanes in time during simulated driving, this issue can be identified through playback and targeted training can be provided. Non-driver students actively participate in training using the simulated driving device, avoiding the distraction caused by passively "riding" in traditional models. For example, on a complex road segment (such as a roundabout or highway exit), student C can familiarize themselves with driving strategies beforehand using the simulation device, which helps with performance during actual driving. Even without actually operating a vehicle, non-driver students can still improve their perception of road conditions and driving judgment through simulation training.

[0043] In summary, the simulated driving training method based on actual road conditions provided in the above embodiments acquires simulated training operation instructions and associated timestamps received by other driving simulation clients within the same vehicle during actual driving training in the driver's seat. These simulated training operation instructions are simulated driving operations performed by other learners within the same vehicle based on the current actual road conditions. During actual driving training in the driver's seat, video data including road condition information is collected. After the driving training ends, the simulated training operation instruction sets of each driving simulation client are merged into the video data based on the timestamps, allowing the instructor to evaluate the simulated driving process of other learners not in the driver's seat. Thus, all learners participate in operational training under actual road conditions, whether in the driver's seat or not, reducing "idle time" for learners. By repeatedly practicing specific road sections (such as hill starts, turns, lane changes, etc.) through the simulation equipment, learners not in the driver's seat can become familiar with handling complex scenarios. The instructor can analyze the performance of all learners through playback, identify problems, and adjust the training content accordingly. All learners actively participate in the training process, creating a mutual learning atmosphere and enhancing the overall training effect.

[0044] In one embodiment, it further includes:

[0045] When conducting actual driving training in the driver's seat, the collected simulated training operation commands are matched with ideal operation commands. If the simulated training operation commands and ideal operation commands do not match, a problem operation prompt message is generated and displayed on the corresponding simulation client.

[0046] Understandably, to further enhance the effectiveness of simulated driving training, an instant feedback mechanism based on matching ideal operating commands has been added. By comparing the trainee's actions on the simulated driving equipment with preset ideal operating commands, a problem operation prompt message is generated when a deviation is detected and displayed on the corresponding simulation client, providing real-time guidance for the trainee to improve their operation.

[0047] For example, trainees perform real-time operations (such as steering, braking, and acceleration) through the driving simulator client, and the system records each operation and its timestamp. For instance, the current trainee's operation is "turn left 15°", with an operation time of T1. Ideal operation instructions are determined in two ways: Preset rules: Ideal operations are defined for standard driving scenarios (such as straight roads, turns, and lane changes). For example, if the current vehicle speed is 50 km / h and a left turn is required 100 meters ahead, the ideal instruction is "turn left at a 30° angle 20 meters in advance." Or, real-time calculation: Ideal operations are dynamically generated based on vehicle status and road conditions (such as distance to the vehicle ahead and curve radius). For example, if the vehicle is traveling at 60 km / h and the distance to the vehicle ahead is 50 meters, the system generates the ideal operation as "decelerate to 40 km / h." The operation instructions collected by the driving simulator client are matched with the ideal operation instructions in real time to determine whether they conform to specifications, such as angle deviation (e.g., steering angle) being less than a set threshold, reasonable speed changes, and adequate braking advance. If the simulated driving instructions do not match the ideal instructions, the system generates a problem message and displays it on the corresponding simulation client. The message includes descriptions of the specific problem, such as "steering angle too small" or "braking too late." It can also include suggestions for improvement, such as "reduce speed to 40 km / h in advance." The message provides real-time feedback to the learner via text, icons, or voice, helping them immediately correct the problematic operation. For example: "Currently understeering, please increase the steering angle to 30°." or "Bracing too late, please begin decelerating 50 meters ahead." This allows learners to receive immediate problem prompts during training, helping them quickly understand the cause of errors and correct their actions, without having to wait for unified feedback after training. The ideal driving instructions are based on actual driving standards, and by comparing them with the learner's actions, learners gradually internalize correct driving habits. Real-time prompts enhance the sense of immersion for non-driver learners in simulated training, allowing them to focus more on the operation and more closely resemble the actual driving experience. Since the problem prompts are automatically generated by the system, instructors do not need to manually point out each learner's mistakes, thus allowing them to focus more on overall guidance.

[0048] In one embodiment, the ideal operating command is determined based on the video data analysis.

[0049] For example, ideal driving instructions are no longer fixed, preset rules or simply calculated based on vehicle status, but are dynamically generated based on collected video data. This method generates ideal driving instructions that are more closely aligned with real-world scenarios through a detailed understanding of actual road conditions, improving the relevance and practicality of training. Therefore, ideal instructions based on video data analysis can reflect current road conditions in real time, dynamically adapting to different driving scenarios, and are closer to reality than fixed rules. Based on the video analysis results, the system generates ideal operations for complex scenarios (such as lane changes, roundabouts, rainy weather, etc.), helping trainees better cope with special situations. Trainees receive video analysis prompts based on real-time road conditions during operation, enabling them to better adapt to dynamically changing driving environments. The ideal driving instructions generated by video analysis serve as an evaluation benchmark, helping instructors quickly identify trainees' weaknesses and provide specific improvement suggestions.

[0050] In one embodiment, it further includes:

[0051] In the case of actual driving training in the driver's seat, the teaching simulation operation instructions and the timestamp associated with the teaching simulation operation instructions received by other teaching simulation clients in the same vehicle are obtained. The teaching simulation operation instructions are simulated driving operations performed by the instructor in the same vehicle as the driver's seat based on the current actual road conditions of the vehicle.

[0052] In the case of actual driving training in the driver's seat, the collected simulated training operation instructions are matched with the teaching simulated operation instructions. If the simulated training operation instructions and the teaching simulated operation instructions do not match, a problem operation prompt message is generated and displayed on the corresponding simulation client.

[0053] Understandably, instructors participate in driving simulations through a teaching simulation client, providing what they deem the most reasonable driving actions (i.e., teaching simulation instructions) based on real-time road conditions. The system matches the student's simulated driving instructions with the instructor's instructions in real time, generating problem operation prompts for any mismatches to guide the student in adjusting their actions immediately. This method, by incorporating the instructor's guidance, provides students with more targeted and authoritative operational references.

[0054] For example, the instructor operates a teaching simulation client (such as a steering wheel, pedals, steering mechanism, etc.) to simulate ideal driving behavior. The system records the instructor's operation commands (such as steering angle, acceleration, or braking force) and their timestamps to ensure synchronization. The student operates through a driving simulation client, and the system records their operation commands and timestamps in real time. The system binds the instructor's and student's operation commands to the same timeline, ensuring that their commands can be matched and compared based on the same actual road conditions. Matching rules can be set according to specific driving tasks, for example: Steering matching: the error between the student's steering angle and the instructor's command angle is less than a set threshold (such as ±5°). Speed ​​matching: the error between the student's acceleration or deceleration operation and the instructor's operation is within a reasonable range (such as ±5 km / h). Braking matching: the difference between the student's braking force and the instructor's braking force is less than a set value. If the student's operation does not match the instructor's command, the system records the type, degree, and time of the deviation. When matching fails, the system generates a prompt message based on the deviation, which may include: describing the student's specific operation problem, such as "oversteering," "understood braking," or "unreasonable speed control." Alternatively, it can provide operational improvement methods, such as "Please reduce the steering angle to 20°" or "Apply the brakes lightly in advance." The prompts are displayed in real-time through the simulation client and can include: text prompts such as "Currently oversteering, please adjust the steering angle to 15°."; ​​graphic prompts: dynamically displaying the deviation direction and suggestions in icon form; and voice prompts such as "Accelerating too quickly, please slow down." Instructors can compare the simulated operational instructions with those of the trainees by replaying the training simulation, providing targeted guidance. For example, in a complex lane-changing maneuver, the trainee turned too early and failed to control the speed. Through replay, the instructor can point out the problem and provide further operational guidance. The replay content can include: the instructor's ideal operation and the trainee's actual operation curve or a video of actual road conditions and the corresponding operation times. Thus, by introducing the instructor's operation as the ideal operational instruction, trainees can receive more targeted and credible guidance. For example, in complex road sections (such as lane changes and roundabouts), the instructor's real-time operation can help trainees understand the best reactions of professional drivers. Instant problem prompts help trainees quickly understand problems and adjust their operations without waiting for centralized feedback after training. For example, if a student fails to slow down in time before a crosswalk, the system prompts "Please lightly apply the brakes in advance," allowing the student to adjust immediately. The instructor's actions can respond in real-time to complex road conditions (such as sudden braking by the vehicle ahead or traffic congestion), generating dynamic and ideal instructions adaptable to various scenarios. For instance, when entering a highway ramp, the instructor's actions can provide the student with direct references regarding steering angle and speed control. By comparing their own actions with the instructor's, students gradually learn how to make more reasonable decisions in different driving situations.

[0055] In one embodiment, it further includes:

[0056] After the driving training is completed, the simulated driving process of other students who are not in the driver's seat is evaluated based on the matching results of the fusion of the simulated training operation command sets of each driving simulation client.

[0057] Understandably, by matching and analyzing the operational commands of learners not in the driver's seat within the driving simulator client with ideal operational commands (or the instructor's simulated teaching commands), a quantitative evaluation result based on operational deviations can be generated. This method can provide learners with comprehensive feedback on their simulated driving operations after training, helping them understand their performance and develop personalized improvement plans.

[0058] For example, learners can clearly understand their specific problems in simulated driving and receive targeted improvement suggestions. For instance, if a learner frequently deviates from the ideal turning angle, the evaluation report might reveal that the main problem is excessively fast turning, suggesting more practice in slow-speed turning. Learners not in the driver's seat also receive detailed evaluations similar to those of learners in actual driving positions, avoiding the negative impact of "passive observation" on learning outcomes. For example, through a comprehensive scoring system, each learner's performance is quantified, ensuring fairness in evaluation. Instructors can quickly identify learners' weaknesses based on their evaluation reports and optimize their teaching strategies. For example, if most learners score low when changing lanes at high speeds, instructors can focus on explaining lane-changing techniques in subsequent training. The operation of the simulated driving client is no longer merely an auxiliary training tool but becomes an important basis for evaluation, encouraging learners to participate more actively.

[0059] According to some embodiments, the actual driving training route for each student in the same vehicle is part of a complete driving training route. The complete driving training route can be obtained by splicing together the actual driving training routes for all students in the same vehicle. The method further includes:

[0060] By combining the evaluation of the student's simulated driving process when not in the driver's seat with the evaluation of the student during the actual driving training process, the overall driving evaluation of the student who independently completes the entire training route is predicted.

[0061] Understandably, each student only undergoes actual driving training on a portion of the complete training course, which is composed of segments from the actual driving experiences of all students within the same vehicle. To predict each student's overall driving ability, the following two evaluation metrics are used: Student's simulated driving process evaluation: scores and evaluations generated based on simulated operation data when the student is not in the driver's seat. Student's actual driving training process evaluation: scores and evaluations generated based on real operation data when the student is in the actual driver's seat. By integrating these two evaluation results and combining them with road condition information from the complete training course, the overall driving performance of the student when independently completing the complete training course is predicted.

[0062] For example, the system records the student's operational data through the simulation client when not in the driver's seat, which may include: steering angle, braking force, acceleration depth, and other operational commands. It also records the timestamps corresponding to the operations and the automatically generated deviation analysis results. The system also records the student's actual operational data in the driver's seat, which may include: operational accuracy, reaction timeliness, and overall consistency. Road condition information during driving (such as lane curvature, dynamic obstacles, etc.) is also recorded. Each student's actual driving segments are sequentially spliced ​​along the timeline to generate a complete driving training segment. All associated road condition information and operational requirements are preserved during segment splicing. The system progressively matches the student's simulated driving operations with ideal operational commands, generating the following key indicators: Operational accuracy score: the proportion of students whose operations deviate little from their ideal operations. Reaction timeliness score: the student's response time to complex road conditions (such as the braking of the vehicle in front, traffic lights). Scenario adaptability score: the student's performance in different scenarios (such as highways, curves). For example, a simulated driving evaluation score of 85 points (good). A comprehensive evaluation of trainees' performance during actual driving is conducted, calculating the following key indicators: Operational smoothness: the smoothness of braking, acceleration, and steering. Scenario performance: the quality of operation in specific road conditions (such as lane changing and turning). Safety score: whether dangerous maneuvers exist (such as sudden braking or failure to use turn signals). For example, an actual driving evaluation score of 78 points (good). The trainee's simulated driving evaluation and actual driving evaluation are combined to predict their driving performance on the complete training route. A weighted model is used to balance the contributions of simulated and actual driving evaluations to the comprehensive evaluation. The weight values ​​can be adjusted according to the training objectives. The comprehensive evaluation score = α × simulated driving score + β × actual driving score, where α and β are weighting coefficients (e.g., 0.4 and 0.6). The comprehensive score can be corrected according to the complexity of different sections within the complete route. For example, highway sections have a higher weight, while simple straight sections have a lower weight. The trainee's driving performance level on the complete route can be classified based on the comprehensive score. Thus, even though trainees only complete a portion of the actual driving training, their comprehensive performance on the complete route can be effectively predicted by combining simulated driving data. The comprehensive evaluation combines trainees' simulated and real-world performance, providing more targeted and holistic feedback. Through predictions and reports, instructors can more effectively plan the next phase of training, improving teaching efficiency. Trainees, through the evaluation reports, clearly understand their strengths and weaknesses, increasing their confidence in driving the entire route.

[0063] According to some embodiments, it also includes:

[0064] During actual driving training in the driver's seat, the instructor's voice information for teaching corrections is collected from inside the vehicle.

[0065] If no corrective teaching voice is collected within a preset time after actual driving operation in the driver's seat, the ideal operating instruction is determined based on the actual driving operation; otherwise, the ideal operating instruction is determined based on the corrective teaching voice information.

[0066] Understandably, in driving training, the instructor's real-time voice correction provides authoritative guidance, directly reflecting the ideal operation in a specific scenario. This embodiment dynamically generates ideal operation instructions by collecting the instructor's corrective voice information inside the vehicle and combining it with the student's actual operation, ensuring that the ideal instructions are highly consistent with the scenario and the instructor's teaching strategy. If the instructor's voice information is not collected within a set time after actual driving, the system automatically generates ideal operation instructions based on the actual driving operation.

[0067] For example, during driving training, the instructor's voice information is collected through an in-vehicle microphone.

[0068] The voice information includes: corrective content, such as "brake in advance" or "decelerate and turn right"; a timestamp corresponding to the specific time of the voice correction, synchronized with the student's operation and road conditions; and voice acquisition supporting noise filtering and keyword recognition. It collects the student's operational data in the driver's seat (such as steering angle, braking force, speed, etc.) and its timestamp. It also collects road condition video through an in-vehicle camera, including dynamic traffic scenes (such as traffic lights, pedestrians, and the status of vehicles ahead). If the instructor's voice information is collected within a preset time (e.g., 5 seconds) after the student's actual driving, ideal operating instructions are generated based on the instructor's corrective content. Semantic analysis is performed on the voice information to extract keywords (such as "brake," "decelerate," "turn left"). The voice content is matched with the driving scenario, and specific ideal operations are generated based on road conditions and vehicle status. If no instructor's voice correction is collected, ideal operating instructions are generated based on actual driving operations and road condition data. Ideal operations are generated based on the vehicle's dynamic status (such as speed and direction) combined with standard driving specifications. Finally, the instructor's voice is integrated with the student's operations and road conditions to form ideal operating instructions. If the coach's voice correction matches the student's action, the system verifies whether the student's action has achieved the desired effect. If the coach's voice correction does not match the student's action, a problem prompt is generated, and the student's deviation is recorded. Thus, the coach's voice correction information directly participates in the generation of ideal instructions, ensuring consistency with training objectives and providing highly scenario-specific guidance. Even when no voice information is collected, the system can automatically generate ideal operation instructions, ensuring that the student's operation evaluation is not interfered with. By analyzing the match between the coach's voice correction and the student's action, the coach can quickly identify the student's weaknesses and adjust the teaching content accordingly. Through voice correction and system prompts, students can instantly understand and adjust their operation deviations.

[0069] Please see Figure 2One embodiment of the simulated driving training device based on actual road conditions in this application may include:

[0070] The acquisition unit 201 is used to acquire, when actual driving training is being conducted in the driver's seat, the simulation training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulation training operation instructions, wherein the simulation training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle.

[0071] The acquisition unit 202 is used to acquire video data, including road condition information, during actual driving training in the driver's seat.

[0072] Evaluation unit 203 is used to integrate the simulated training operation instructions of each driving simulation client into the video data based on timestamps after the driving training ends, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

[0073] In summary, the simulated driving training device based on actual road conditions provided in the above embodiments acquires simulated training operation instructions and associated timestamps received by other driving simulation clients within the same vehicle during actual driving training in the driver's seat. These simulated training operation instructions are simulated driving operations performed by other learners in the same vehicle based on the current actual road conditions. During actual driving training in the driver's seat, video data including road condition information is collected. After the driving training ends, the simulated training operation instruction sets of each driving simulation client are merged into the video data based on the timestamps, allowing the instructor to evaluate the simulated driving process of other learners not in the driver's seat. Thus, all learners participate in operational training under actual road conditions, whether in the driver's seat or not, reducing "idle time" for learners. By repeatedly practicing specific road sections (such as hill starts, turns, lane changes, etc.) through the simulation device, learners not in the driver's seat can become familiar with handling complex scenarios. Instructors can analyze the performance of all learners through playback, identify problems, and adjust the training content accordingly. All learners actively participate in the training process, creating a mutual learning atmosphere and enhancing the overall training effect.

[0074] above Figure 2 The simulated driving training device based on actual road conditions in the embodiments of this application has been described from the perspective of modular functional entities. The following is a detailed description of the simulated driving training device based on actual road conditions in the embodiments of this application from the perspective of hardware processing. Please refer to [link / reference]. Figure 3 One embodiment of the simulated driving training device 300 based on actual road conditions in this application includes:

[0075] The system includes an input device 301, an output device 302, a processor 303, and a memory 304, wherein the number of processors 303 can be one or more. Figure 3 Taking a processor 303 as an example. In some embodiments of this application, the input device 301, output device 302, processor 303, and memory 304 can be connected via a bus or other means, wherein... Figure 3 Taking the example of a connection between China and Israel via a bus.

[0076] Specifically, by calling the operation instructions stored in memory 304, processor 303 executes the following steps:

[0077] In the case of actual driving training in the driver's seat, the simulation training operation instructions and the timestamp associated with the simulation training operation instructions received by other driving simulation clients in the same vehicle are obtained. The simulation training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle.

[0078] Collect video data, including road condition information, while conducting actual driving training in the driver's seat;

[0079] After the driving training ends, the set of simulated training operation instructions from each driving simulation client is merged into the video data based on timestamps, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

[0080] By calling the operation instructions stored in memory 304, processor 303 is also used to execute... Figure 1 Any of the methods in the corresponding embodiments.

[0081] Please see Figure 4 , Figure 4 A schematic diagram of an embodiment of the electronic system provided in this application.

[0082] like Figure 4 As shown, this application provides an electronic system including a memory 410, a processor 420, and a computer program 411 stored in the memory 420 and executable on the processor 420. When the processor 420 executes the computer program 411, it performs the following steps:

[0083] In the case of actual driving training in the driver's seat, the simulation training operation instructions and the timestamp associated with the simulation training operation instructions received by other driving simulation clients in the same vehicle are obtained. The simulation training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle.

[0084] Collect video data, including road condition information, while conducting actual driving training in the driver's seat;

[0085] After the driving training ends, the set of simulated training operation instructions from each driving simulation client is merged into the video data based on timestamps, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

[0086] In practical implementation, when the processor 420 executes the computer program 411, it can achieve... Figure 1 Any of the corresponding implementation methods in the embodiments.

[0087] Since the electronic system described in this embodiment is the equipment used to implement a simulated driving training device based on actual road conditions in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic system in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic system implements the method in the embodiments of this application will not be described in detail here. As long as those skilled in the art use the equipment to implement the method in the embodiments of this application, it falls within the scope of protection of this application.

[0088] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided in this application.

[0089] like Figure 5 As shown, this embodiment provides a computer-readable storage medium 500 on which a computer program 511 is stored. When the computer program 511 is executed by a processor, it performs the following steps:

[0090] In the case of actual driving training in the driver's seat, the simulation training operation instructions and the timestamp associated with the simulation training operation instructions received by other driving simulation clients in the same vehicle are obtained. The simulation training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle.

[0091] Collect video data, including road condition information, while conducting actual driving training in the driver's seat;

[0092] After the driving training ends, the set of simulated training operation instructions from each driving simulation client is merged into the video data based on timestamps, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

[0093] In practical implementation, when the computer program 511 is executed by the processor, it can achieve the following: Figure 1 Any of the corresponding implementation methods in the embodiments.

[0094] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0095] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0098] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0099] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to perform actions such as... Figure 1The process of the simulated driving training method based on actual road conditions in the corresponding embodiment.

[0100] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0101] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A simulated driving training method based on actual road conditions, characterized in that, include: In the case of actual driving training in the driver's seat, the simulation training operation instructions and the timestamp associated with the simulation training operation instructions received by other driving simulation clients in the same vehicle are obtained. The simulation training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat according to the current actual road conditions of the vehicle. Collect video data, including road condition information, while conducting actual driving training in the driver's seat; After the driving training ends, the set of simulated training operation instructions from each driving simulation client is merged into the video data based on timestamps, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

2. The method according to claim 1, characterized in that, Also includes: When conducting actual driving training in the driver's seat, the collected simulated training operation commands are matched with ideal operation commands. If the simulated training operation commands and ideal operation commands do not match, a problem operation prompt message is generated and displayed on the corresponding driving simulation client.

3. The method according to claim 2, characterized in that, The ideal operating instructions are determined based on the analysis of the video data.

4. The method according to claim 1, characterized in that, Also includes: In the case of actual driving training in the driver's seat, the teaching simulation operation instructions and the timestamp associated with the teaching simulation operation instructions received by other driving simulation clients in the same vehicle are obtained. The teaching simulation operation instructions are simulated driving operations performed by the instructor in the same vehicle as the driver's seat based on the current actual road conditions of the vehicle. When conducting actual driving training in the driver's seat, the collected simulated training operation instructions are matched with the teaching simulated operation instructions. If the simulated training operation instructions and the teaching simulated operation instructions do not match, a problem operation prompt message is generated and displayed on the corresponding driving simulation client.

5. The method according to any one of claims 2 to 4, characterized in that, Also includes: After the driving training is completed, the simulated driving process of other students who are not in the driver's seat is evaluated based on the matching results of the fusion of the simulated training operation command sets of each driving simulation client.

6. The method according to claim 5, characterized in that, The actual driving training route for each student in the same vehicle constitutes part of a complete driving training route. The complete driving training route is obtained by splicing together the actual driving training routes for all students in the same vehicle. The method further includes: By combining the evaluation of the student's simulated driving process when not in the driver's seat with the evaluation of the student during the actual driving training process, the overall driving evaluation of the student who independently completes the entire training route is predicted.

7. The method according to claim 2, characterized in that, Also includes: During actual driving training in the driver's seat, the instructor's voice information for teaching corrections is collected from inside the vehicle. If no corrective teaching voice is collected within a preset time after actual driving operation in the driver's seat, the ideal operating instruction is determined based on the actual driving operation; otherwise, the ideal operating instruction is determined based on the corrective teaching voice information.

8. A simulated driving training device based on actual road conditions, characterized in that, include: The acquisition unit is used to acquire, when actual driving training is being conducted in the driver's seat, the simulated training operation instructions received by other driving simulation clients in the same vehicle and the timestamp associated with the simulated training operation instructions, wherein the simulated training operation instructions are simulated driving operations performed by other trainees in the same vehicle as the driver's seat based on the current actual road conditions of the vehicle. The acquisition unit is used to collect video data, including road condition information, during actual driving training in the driver's seat. An evaluation unit is used to integrate the simulated training operation instructions of each driving simulation client into the video data based on timestamps after the driving training ends, so that the instructor can evaluate the simulated driving process of other students who are not in the driver's seat.

9. An electronic system comprising a memory and a processor, characterized in that, When the processor executes the computer program stored in the memory, it implements the steps of the simulated driving training method based on actual road conditions as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the simulated driving training method based on actual road conditions as described in any one of claims 1 to 7.

Images