Driving training system and driving training data generation method based on v2x data

Abstract

The application relates to the field of data processing and provides a driving training system based on V2X data, which comprises the following: a V2X data deep processing and digital track twin unit, which is used for accessing a V2X historical database, extracting a complex interactive scene segment, generating a digital standard training track and an augmented reality guidance information frame; a multi-agent collaborative training management and dynamic broadcast unit, which is used for maintaining a track library, receiving a vehicle pose in real time, dynamically binding a training track, predicting and detecting potential conflicts of multiple vehicles, dynamically adjusting the track and multicasting guidance information; a distributed augmented reality guidance display array, which is used for rendering and displaying the augmented reality guidance information frame according to the multicast information; and a driving behavior multi-modal acquisition and real-time evaluation feedback unit, which is used for acquiring multi-modal operation data, comparing the multi-modal operation data with an expected operation benchmark model, generating a correction prompt and injecting the guidance information frame. The application realizes high reality and multi-vehicle dynamic collaborative interaction of a driving training scene and improves training safety and evaluation accuracy.

Description

Technical Field

[0001] This invention relates to the field of data processing. Specifically, in the context of driving simulation, this invention relates to a driving training system based on V2X data and a method for generating driving training data. Background Technology

[0002] Current driver training often takes place in training grounds, which are mostly enclosed special areas. For safety reasons, training is not conducted in actual road conditions, which prevents trainees from dealing with complex real-world road situations. As a result, trainees have poor handling skills when encountering different road conditions, which in turn affects their safe driving after obtaining their driver's license.

[0003] To address these issues, virtual training simulators are often used to practice driving on simulators. However, these simulators lack realism and cannot achieve the same effect as practicing in a real car. Therefore, existing driving training methods suffer from low realism and poor training results. Summary of the Invention

[0004] To address the issues of insufficient training realism, limited scenarios, lack of dynamic interaction, and insufficient scientific quantitative evaluation in existing technologies, this application proposes a driving training system and method based on V2X data to safely transfer real traffic scenarios from open roads to closed training grounds, enabling multi-vehicle dynamic collaborative interactive training and real-time accurate evaluation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A driving training system based on V2X data is deployed in a closed training ground equipped with multiple simulated lanes and environmental simulation facilities. The system includes: a V2X data deep processing and digital trajectory twin unit, used to access the urban road network V2X historical database, clean, denoise, and re-identify vehicle trajectories of multi-source heterogeneous traffic flow data, and decouple to generate a high-confidence single-vehicle continuous spatiotemporal trajectory; used to automatically extract complex interactive scene segments including cut-in, emergency braking, and unprotected left turns from the single-vehicle spatiotemporal trajectory based on a driving behavior dynamics model and a key scene mining algorithm; used to register and scale the interactive scene segments with a high-precision 3D point cloud map of the training ground to generate a digital standard training trajectory that is completely mapped to the physical training ground, and synchronously generate augmented reality guidance information frames containing the relative positions of multiple vehicles, traffic signal status, and expected operation guidance for each key decision point of the trajectory; The multi-agent collaborative training management and dynamic broadcasting unit is used to maintain the digital standard training trajectory library and the corresponding augmented reality guidance information frame sequence; to receive in real time the precise pose, speed and control status reported by each training vehicle in the field, which integrates vehicle-mounted GNSS / IMU and UWB positioning data; to dynamically bind and update the training trajectory currently being executed for each training vehicle; to predict the future short-term trajectory of each vehicle based on the kinematic model, and to detect potential conflicts between multiple vehicles in real time according to the predicted trajectory and the preset anti-collision safety envelope; when a potential conflict is detected, to dynamically adjust the subsequent trajectory point sequence or guidance information of the affected vehicle, and to re-plan the broadcast timing. Based on the vehicle's real-time pose, binding trajectory, and collision avoidance decisions, the system calculates the augmented reality guidance information frame to be displayed in the next moment and its corresponding target display device identifier, and performs multicasting via a low-latency communication network. The distributed augmented reality guidance display array consists of multiple display devices with independent identifiers, deployed above or to the side of the lane according to the key location topology in the training field's 3D point cloud map. Each display device listens to the multicast information of the dynamic broadcast unit, and when the target identifier in the multicast information matches the identifier of its own device, it renders and displays the augmented reality guidance information frame in real time, providing the driver with highly immersive visual guidance. The multimodal acquisition and real-time evaluation feedback unit for driving behavior is used to acquire multimodal operation data of the driver through high-precision sensors deployed on the training vehicle, including steering wheel angle / torque, pedal travel / rate, gear signal, and gaze area; and to establish the expected operation benchmark model for each key decision point in the digital standard training trajectory. This is used to perform spatiotemporal alignment and multidimensional comparison of real-time collected multimodal operation data with the expected operation benchmark model of the current decision point, and to calculate trajectory tracking error, operation smoothness and risk indicators; based on the comparison and calculation results, it is used to generate evaluation conclusions and correction prompts in real time, and to inject the correction prompts into the augmented reality guidance information frame to be broadcast by the dynamic broadcast unit in the form of an overlay layer, forming a closed-loop feedback.

[0006] The above solution migrates complex real-world road scenarios to a closed training ground through deep V2X data processing and digital trajectory twin units. It utilizes multi-agent collaborative training management and dynamic broadcasting units to achieve dynamic interaction and safety control of multiple vehicles. Furthermore, it achieves virtual-real fusion and closed-loop training through a distributed augmented reality guidance display array and a multi-modal acquisition and real-time evaluation feedback unit for driving behavior. This solves the problems of existing technologies, such as limited training scenarios, lack of dynamic interaction, and inaccurate evaluation.

[0007] As an optional implementation, the V2X data deep processing and digital trajectory twin unit extracts complex interaction scene segments based on a key scene mining algorithm, including: performing differential processing on the spatiotemporal trajectory of a single vehicle to calculate the time series of its acceleration, jerk, and lateral position offset rate; setting jerk thresholds, lateral offset rate thresholds, and minimum spatiotemporal distance thresholds with surrounding vehicles; when the trajectory point data simultaneously meets the conditions of jerk exceeding the threshold, lateral offset rate exceeding the threshold, and spatiotemporal distance with surrounding vehicles below the threshold, the trajectory segment is marked as a high-risk interaction scene; using a clustering algorithm to classify all high-risk interaction scenes, and selecting the most representative segments from each category according to the teaching syllabus of the training system to form a standard training scene library.

[0008] The above scheme can accurately identify high-risk, high-value interaction scenarios by performing differential processing and multi-dimensional threshold determination on trajectory data, thereby improving the typicality and effectiveness of training scenarios.

[0009] As an optional implementation, the multi-agent collaborative training management and dynamic broadcasting unit predicts conflicts and dynamically adjusts them based on kinematic models, including: using the current state of each vehicle as the initial value, predicting the trajectory within the next 3-5 seconds using a simplified single-vehicle kinematic model to form a predicted trajectory cluster; defining a three-dimensional dynamic safety envelope for each vehicle that includes its external dimensions and safety margin, and expanding it along its predicted trajectory cluster; detecting in real time whether the expanded safety envelopes of any two vehicles intersect in the spatiotemporal domain; if an intersection is detected, determining that there is a potential conflict, immediately performing local replanning on the digital standard training trajectory bound to the following vehicle or the vehicle logically obligated to yield, generating a conflict-free corrected trajectory segment by inserting deceleration points, modifying path points, or increasing waiting time, and updating the subsequent guidance information frame sequence.

[0010] The above scheme, through kinematic model prediction and dynamic safety envelope expansion detection, enables early identification and dynamic avoidance of potential conflicts during multi-vehicle collaborative training, thus ensuring the safety of multi-vehicle interactive training in a closed training ground.

[0011] As an optional implementation, the multi-agent collaborative training management and dynamic broadcasting unit is also used to realize flexible grouping and dynamic entry of training vehicles, including: setting a virtual fleet training mode, allowing the administrator to specify or the system to automatically generate a group of training vehicles with logical following relationships; providing two modes for newly applying training vehicles to enter the field: single-vehicle free training mode, or grouping training mode to join a specified virtual fleet; in the grouping training mode, dynamically assigning a fleet sequence number to a new vehicle, and the digital standard training trajectory bound to it will be dynamically adjusted according to the real-time position and speed of the fleet leader to maintain the fleet formation and safe following distance; broadcasting the relative position and status of vehicles in the fleet in real time as part of the augmented reality guidance information for reference by team vehicles.

[0012] The above scheme enriches the training mode through flexible grouping and dynamic entry mechanism, which can simulate the following and convoy driving scenarios in real traffic flow, and improves the systematicness and realism of the training.

[0013] As an optional implementation, the system further includes a high-precision field-end positioning and perception subsystem, comprising: multiple UWB positioning base stations deployed at key locations in the training field for communicating with UWB tags on the training vehicles to achieve centimeter-level real-time positioning of the vehicles within the field; roadside vision sensors deployed near lane lines for capturing detailed behaviors such as vehicle crossing lines and parking positions, and fusing them with UWB positioning data to provide higher-precision pose estimation; and a multi-agent collaborative training management and dynamic broadcasting unit that fuses the UWB positioning data and roadside vision data as redundant verification and optimization input for the real-time pose information of the training vehicles, improving the accuracy of trajectory tracking and collision detection.

[0014] The above scheme achieves centimeter-level high-precision positioning through UWB and vision fusion positioning, providing a precise data foundation for trajectory tracking and collision detection, and further improving the security and evaluation accuracy of the training system.

[0015] As an optional implementation, the driving behavior multimodal acquisition and real-time evaluation feedback unit performs multi-dimensional comparison and risk assessment, including: trajectory tracking error dimension: calculating the root mean square error of lateral deviation and heading deviation between the actual vehicle trajectory and the digital standard training trajectory; Operational smoothness dimension: Analyze the power spectrum of the steering wheel angle change rate and pedal travel change rate to evaluate the smoothness and predictability of the operation; Risk indicator dimension: Combine real-time vehicle speed, minimum distance to surrounding virtual or real obstacles, TTC and other parameters to calculate the comprehensive risk coefficient at the current moment; The real-time generated correction prompts will be generated in a targeted manner according to the main deviation dimensions, including virtual lane line guidance for correcting lateral deviation, "smooth steering" text for prompting overly aggressive operation, and a conspicuous red flashing border warning when the risk coefficient exceeds the threshold.

[0016] The above-mentioned solution achieves accurate diagnosis and timely feedback on driving operations through multi-dimensional quantitative assessment and targeted corrective prompts, which helps trainees quickly develop correct driving habits and stress responses.

[0017] As an optional implementation, the system further includes a training result quantitative evaluation and report generation unit, used for: summarizing all evaluation data from the multimodal acquisition and real-time evaluation feedback unit of driving behavior throughout the entire trajectory cycle after a training task is completed; calculating the driver's scores and total score on multiple sub-items such as trajectory tracking accuracy, operational smoothness, rule compliance, and risk avoidance ability based on a preset weighting system; automatically generating a structured training report, including a score summary, key event replay, analysis of strengths and areas for improvement, and supporting trend analysis by comparing with historical training records.

[0018] The above solution generates structured quantitative evaluation reports, enabling visualization and data-driven management of training effectiveness, which facilitates trainees and coaches to fully grasp training progress and weaknesses.

[0019] As an optional implementation, the system further includes an immersive environment simulation execution unit, comprising: a controllable spray device deployed on a specific road section to simulate low-adhesion road surfaces or vision interference scenarios such as rain, water splashing from vehicles ahead; an intelligent light and shadow projection device deployed on the roadside to simulate light change scenarios such as glare from oncoming vehicle high beams and transitions between light and dark in tunnels; and a multi-agent collaborative training management and dynamic broadcasting unit that can send trigger commands to the environment simulation execution unit when the vehicle reaches a specific location based on scene tags in the digital standard training trajectory, thereby reproducing the real environmental challenges in the V2X data source within the physical training field.

[0020] The above solution uses a physical environment simulation device to reproduce real-world environmental challenges such as rain, fog, and glare in a closed space, further enhancing the immersion and scene coverage of the training.

[0021] As an optional implementation, the display device in the distributed augmented reality guidance display array is a transparent LED screen or a laser projection device; when a transparent LED screen is used, it is installed above the lane, and the displayed augmented reality guidance information frame can be fused and superimposed with the real road background outside the vehicle seen by the driver; when a laser projection device is used, it projects the guidance information directly onto the training ground road surface in front of the vehicle, forming a virtual guidance sign that is integrated with the real environment.

[0022] The above solution achieves seamless integration of virtual information with the real environment through transparent LED screens or laser projection equipment, enhancing the realism and immersion of visual guidance.

[0023] This invention also provides a driving training method based on V2X data, applied to the system described above. The method includes: a digital twin trajectory construction step: processing historical V2X data, mining typical high-risk interaction scenarios, generating a digital standard training trajectory mapped to a physical training field, and a synchronized augmented reality guidance information sequence; and a multi-agent collaborative management step: dynamically allocating and binding training trajectories to multiple training vehicles, predicting trajectory and detecting conflicts in real time, dynamically adjusting the planning to ensure the safety of collaborative training, and calculating and publishing guidance information. Augmented Reality Guidance Steps: Augmented reality guidance information is dynamically presented based on the vehicle's real-time location using a distributed display array, providing immersive visual guidance; Closed-Loop Evaluation and Feedback Steps: Multimodal acquisition of driving behavior is performed, compared and evaluated in real-time with digital twin standards, and corrective prompts are integrated into the guidance information in real-time, achieving closed-loop training of "guidance-operation-evaluation-correction"; Environmental Simulation and Report Generation Steps: A physical environment simulation device is triggered to reproduce complex scenarios, and a quantitative evaluation report and trend analysis are generated after training. The above methods and steps correspond to the system units and also have the beneficial effects of improving training realism, realizing dynamic collaborative interaction, and accurate evaluation.

[0024] This invention transforms complex traffic scenarios on real roads into standard training trajectories within a closed training field through deep V2X data processing and a digital trajectory twin unit, achieving a high degree of realism and richness in the training scenario. Through multi-agent collaborative training management and a dynamic broadcasting unit, it supports dynamic interactive training with multiple vehicles on site, overcoming the limitations of traditional "single-vehicle training" and ensuring training safety through real-time conflict detection and dynamic adjustment. By integrating virtual traffic elements with the real physical environment through a distributed augmented reality guidance display array, a highly immersive visual guidance is provided. Through multimodal acquisition and real-time evaluation feedback units of driving behavior, real-time and accurate evaluation and correction feedback of driving operations are achieved, forming an efficient training loop. In summary, this invention constructs a safe, realistic, intelligent, and evaluable next-generation driving training system, effectively improving the quality of driver training and road traffic safety. Attached Figure Description

[0025] Figure 1 This is a schematic diagram illustrating the composition of a V2X-based driving training system in one embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram illustrating the steps of a driving training method based on V2X data in one embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0029] Example 1:

[0030] like Figure 1 As shown, this embodiment provides a driving training system based on V2X data, which is deployed in a closed training ground equipped with multiple simulated lanes and environmental simulation facilities. Figure 1 As shown, the system constructs a complete technical architecture from real data input to virtual-real fusion execution and then to closed-loop feedback evaluation. It mainly includes four core components: V2X data deep processing and digital trajectory twin unit 101, multi-agent collaborative training management and dynamic broadcasting unit 102, distributed augmented reality guidance display array 103, and driving behavior multimodal acquisition and real-time evaluation feedback unit 104.

[0031] The V2X data deep processing and digital trajectory twin unit, serving as the system's data source and core for scene generation, is used to access the urban road network V2X historical database. It cleans, denoises, and re-identifies vehicle trajectories from multi-source heterogeneous traffic flow data, decoupling to generate high-confidence continuous spatiotemporal trajectories for individual vehicles. Specifically, the urban road network V2X historical database contains massive amounts of real traffic data from diverse sources and formats, such as vehicle position sequences captured by roadside sensing devices, BSM messages broadcast by vehicle-mounted OBUs, and phase status data from traffic light controllers. This unit first cleans the raw data, removing packet loss and abnormal transition points caused by signal obstruction. Subsequently, a trajectory re-identification algorithm is used to associate discrete perception data fragments into continuous single-vehicle trajectories, thereby reconstructing the entire vehicle driving process on a real road in digital space. This unit is also used to automatically extract complex interaction scene fragments, including cut-in, emergency braking, and unprotected left turns, from the single-vehicle spatiotemporal trajectory based on a driving behavior dynamics model and a key scene mining algorithm. It should be understood that the scene types mentioned here are merely examples; the system can extract other high-risk scenarios, such as sudden pedestrian appearances and forced lane changes, by adjusting the parameters of the mining algorithm according to actual training needs.

[0032] This unit is further used to register and scale the interactive scene fragments with the high-precision 3D point cloud map of the training field, generate a digital standard training trajectory that is completely mapped to the physical training field, and synchronously generate augmented reality guidance information frames containing the relative positions of multiple vehicles, traffic signal status, and expected operation guidance for each key decision point of the trajectory.

[0033] Through registration and scaling, a cut-in scene from a real-world intersection in Beijing can be precisely "transferred" and mapped to a specific lane in a closed training area, achieving a realistic reproduction of the training scene. Augmented reality guidance frames not only contain the image data required for visual rendering but also carry metadata such as timestamps, spatial coordinates, and trigger conditions, providing a data foundation for subsequent virtual-real fusion display.

[0034] The multi-agent collaborative training management and dynamic broadcasting unit, as the central scheduling core of the system, is used to maintain the digital standard training trajectory library and the corresponding augmented reality guidance information frame sequence. This unit is equivalent to a dynamic director, responsible for coordinating the operational logic of all training vehicles in the training area. It is used to receive in real time the precise pose, velocity, and control status reported by each training vehicle in the training area, which integrates onboard GNSS / IMU and UWB positioning data.

[0035] By fusing multi-source positioning data, the system can acquire centimeter-level location information of vehicles within the training field, a prerequisite for precise guidance and safe control. This unit is used to dynamically bind and update the current training trajectory for each training vehicle. When a vehicle enters the training field, the system automatically assigns a digitized standard training trajectory according to its training plan; if an emergency occurs during training, the system can also dynamically update the bound trajectory. This unit is used to predict the future short-term trajectories of each vehicle based on kinematic models, and to detect potential conflicts between multiple vehicles in real time based on the predicted trajectories and the preset collision avoidance safety envelope.

[0036] The collision avoidance safety envelope is a three-dimensional virtual space region that surrounds the vehicle and dynamically expands with increasing vehicle speed, providing a safe braking distance. When a potential collision is detected, this unit dynamically adjusts the subsequent trajectory point sequence or guidance information of the affected vehicle and re-plans the broadcast timing, thereby physically eliminating the risk of collision. This unit also calculates the augmented reality guidance information frame to be presented next, along with its corresponding target display device identifier, based on the vehicle's real-time pose, bound trajectory, and collision avoidance decisions, and multicasts it via a low-latency communication network. The multicast mechanism ensures that the guidance information is accurately and efficiently sent to the target display device, avoiding network congestion.

[0037] The distributed augmented reality guidance display array consists of multiple display devices with independent identifiers, deployed above or to the sides of lanes based on key location topologies in the training field's 3D point cloud map. These display devices are typically deployed at key decision points such as intersection entrances, curve start points, and merging zones, forming a visual guidance network covering the entire field. Each display device listens to multicast information from the dynamic broadcast unit. When a target identifier in the multicast information matches the device's identifier, it renders and displays the augmented reality guidance information frame in real time, providing drivers with highly immersive visual guidance. For example, when a vehicle approaches a simulated intersection, the display device ahead renders a virtual image of oncoming traffic in real time. This image blends perfectly with the real road background seen by the driver through the windshield, creating a realistic visual experience of traffic flow.

[0038] The multimodal driving behavior acquisition and real-time evaluation feedback unit, acting as the system's "intelligent coach," collects multimodal operational data from the driver using high-precision sensors deployed on the training vehicle. This data includes steering wheel angle / torque, pedal travel / rate, gear position signal, and gaze area. This comprehensive data record the driver's operational details and psychological state. This unit is used to establish a benchmark model for the expected operation at each key decision point in the digital standard training trajectory.

[0039] The expected operational baseline model is generated based on the operational data of professional drivers and represents standard and safe driving behavior norms. This unit is used to perform spatiotemporal alignment and multidimensional comparison between the real-time collected multimodal operational data and the expected operational baseline model at the current decision point, calculating trajectory tracking error, operational smoothness, and risk indicators. Through comparison, the system can accurately identify the trainee's operational deviations, such as delayed steering timing or excessive braking.

[0040] This unit generates evaluation conclusions and correction prompts in real time based on comparison and calculation results. These correction prompts are then overlaid into the augmented reality guidance information frame to be broadcast by the dynamic broadcast unit, forming a closed-loop feedback. For example, when the system detects insufficient steering input from the trainee, a virtual arrow pointing to the correct driving route is overlaid on the guidance screen, correcting the trainee's erroneous operation in real time. This constructs an efficient training closed loop of "guidance-operation-evaluation-correction."

[0041] Through the above architecture, this embodiment realizes the safe transfer of complex traffic scenarios from real roads to a closed training ground, and ensures the safety of multi-vehicle interactive training through multi-agent collaborative management. Finally, the closed-loop feedback mechanism effectively improves the practical effect and teaching quality of driving training.

[0042] Example 2:

[0043] Based on Example 1, this example provides a detailed description of the specific algorithm process for deep V2X data processing and extraction of complex interactive scene fragments using digital trajectory twin units.

[0044] The V2X data deep processing and digital trajectory twin unit extracts complex interaction scene fragments based on key scene mining algorithms. Specifically, this includes: performing differential processing on the spatiotemporal trajectory of a single vehicle to calculate the time series of its acceleration, jerk, and lateral position offset rate. In detail, the original V2X trajectory data is usually a series of latitude and longitude coordinates with timestamps. Although it records the vehicle's position, it cannot directly reflect the driver's operational intentions and risk level.

[0045] This embodiment processes position data using a numerical differentiation algorithm. Velocity is the first derivative of position, acceleration is the second derivative, and jerk (also known as jolt) is the rate of change of acceleration, i.e., the third derivative of position. Jerk can sensitively detect abrupt changes in driver input, such as sudden braking or sharp steering; its unit is typically m / s². 3 Lateral position offset reflects the vehicle's swaying or lane-changing tendency within the lane. By calculating the time series of these dynamic parameters, the system transforms monotonous trajectory points into a dynamic data stream rich in behavioral characteristics.

[0046] Subsequently, the system sets the aggression threshold, lateral deviation rate threshold, and minimum spatial distance threshold with respect to surrounding vehicles. In this embodiment, based on the general driving patterns on urban roads, the aggression threshold can be set to 3 m / s. 3 The lateral offset threshold can be set to 0.5 m / s, and the minimum spatiotemporal distance threshold can be set to 5 meters. It should be understood that these thresholds are not fixed and can be dynamically adjusted by the system administrator based on the training difficulty level or the actual road environment. For example, for novice training mode, the jerk threshold can be lowered to 2 m / s. 3 This allows for the selection of more operational details that require attention; however, in high-speed scene training, the minimum spatiotemporal distance threshold needs to be increased accordingly.

[0047] When trajectory point data simultaneously meets the criteria of exceeding a threshold for swiftness and lateral offset, and is within a certain spatial and temporal distance from surrounding vehicles, the trajectory segment is marked as a high-risk interaction scenario. This multi-dimensional joint judgment mechanism can effectively eliminate misjudgments caused by a single factor and accurately pinpoint the actual dangerous interaction moment.

[0048] For example, when a vehicle performs a sudden braking test on an open road, although the abruptness is high, the system determines it is not a high-risk interaction scenario because the spatial and temporal distance from surrounding vehicles is relatively large. Similarly, when a vehicle is slowly following another vehicle in a congested area, although the distance to surrounding vehicles is close, the abruptness is low, which is considered normal driving. Only when a vehicle simultaneously experiences a sudden change in operation (high abruptness), abnormal posture (large lateral deviation), and is at a dangerous distance (close distance) is it identified by the system as a high-risk interaction scenario, such as emergency avoidance or forced lane change.

[0049] Finally, the system uses clustering algorithms to categorize all high-risk interaction scenarios and, based on the training system's syllabus, selects the most representative segments from each category to form a standard training scenario library. Specifically, the system can use clustering algorithms such as K-means or DBSCAN to divide the massive number of segments into categories such as "intersection left-turn conflict," "highway cut-in," and "ghost pedestrian" based on attributes such as the scene's speed characteristics, the number of interacting objects, and road geometry. Subsequently, the system calculates the distance between each segment and the cluster center, selecting the segment with the closest distance and most typical characteristics as the standard training scenario. This process extracts high-value teaching materials from massive amounts of raw data, ensuring the relevance and effectiveness of the training content and avoiding the subjectivity and inefficiency of manual selection.

[0050] Example 3:

[0051] Based on Example 1, this example elaborates on the specific implementation mechanism of multi-agent collaborative training management and dynamic broadcasting unit based on kinematic model to predict conflicts and make dynamic adjustments.

[0052] This unit uses the current state of each vehicle as an initial value and a simplified single-vehicle kinematics model to predict the trajectory within the next 3-5 seconds, forming a cluster of predicted trajectories. Specifically, the simplified single-vehicle kinematics model can be expressed as: x(t+Δt) = x(t) + v(t)·cos(ψ(t))·Δt, y(t+Δt) = y(t) + v(t)·sin(ψ(t))·Δt, where x and y are the coordinates of the vehicle's center of mass, v is the vehicle speed, ψ is the heading angle, and Δt is the time step. The system iteratively calculates the set of possible locations the vehicle may reach within the next 3 to 5 seconds in 50-millisecond steps.

[0053] It should be understood that, considering the physical limitations of the vehicle's steering system, a maximum steering angular velocity constraint can be introduced into the model to make the predicted trajectory more consistent with the actual dynamic characteristics of the vehicle and avoid generating unexecutable "ghost trajectories." The predicted trajectory cluster is actually a set of possible paths, covering multiple possibilities such as the vehicle continuing straight, turning left, and turning right, thus providing a redundant safety judgment space for conflict detection.

[0054] Subsequently, the system defines a three-dimensional dynamic safety envelope for each vehicle, including its external dimensions and safety margin, and expands it along its predicted trajectory cluster. In this embodiment, the three-dimensional dynamic safety envelope not only includes the vehicle's length, width, and height geometry but also introduces a safety margin. The size of the safety margin is not a fixed value but is proportional to the vehicle's speed. For example, the safety margin L_margin can be set as k·v, where k is the expansion coefficient (e.g., a time equivalent of 0.5 seconds), and v is the current vehicle speed. This means that when a vehicle is traveling at a speed of 20 m / s, an additional 10 meters of safety buffer distance will be added in front of it.

[0055] This dynamic expansion mechanism simulates the physical law in real driving that "the faster the vehicle speed, the longer the braking distance required." The three-dimensional dynamic safety envelope is spatially represented as an irregular cylinder extending along the trajectory, whose cross-sectional size dynamically adjusts with changes in speed.

[0056] The system continuously monitors whether the expanded safety envelopes of any two vehicles intersect in the spatiotemporal domain. This is essentially a four-dimensional spatiotemporal collision detection problem (three-dimensional space + time dimension). The system aligns the safety envelopes of the two vehicles on the time axis and calculates whether the two three-dimensional bodies geometrically overlap at any time t. If an intersection is detected, a potential conflict is determined. This method is more scientific than traditional distance-based thresholding because it comprehensively considers the vehicle's geometric dimensions, motion trends, and the cumulative risk from speed.

[0057] Once a potential conflict is identified, the system immediately performs local replanning on the digital standard training trajectory associated with the following vehicle or the vehicle logically obligated to yield. This is done by inserting deceleration points, modifying waypoints, or increasing waiting time to generate a conflict-free corrected trajectory segment, and then updating the subsequent guidance information frame sequence. For example, in a simulated intersection scenario, the predicted safety envelopes of vehicle A and vehicle B intersect in the central area of ​​the intersection. Based on traffic rules, the system determines that vehicle B must yield and subsequently inserts a deceleration point into vehicle B's trajectory planning.

[0058] The specific operation is as follows: 1.5 seconds before the conflict occurs, the target speed of vehicle B is smoothly reduced from 15 m / s to 5 m / s, and a corresponding braking curve is generated. This corrected trajectory segment replaces the corresponding segment in the original trajectory. Simultaneously, the system regenerates augmented reality guidance information frames, updating the previously displayed "Maintain Speed" instruction to a virtual "Decelerate and Observe" sign, thus providing dual protection for training safety at both the physical and informational levels. Through this conflict resolution mechanism based on model prediction and dynamic envelope, the system can complete the safe scheduling of multi-vehicle collaboration within milliseconds, ensuring absolute safety for multi-vehicle interactive training within a closed training ground.

[0059] Example 4:

[0060] Based on Example 1, this example elaborates on the specific implementation method of the multi-agent collaborative training management and dynamic broadcasting unit to realize the flexible grouping and dynamic entry of training vehicles.

[0061] The multi-agent collaborative training management and dynamic broadcasting unit is also used to realize flexible grouping and dynamic entry of training vehicles. In practical applications, independent training of a single vehicle is often difficult to simulate complex scenarios such as following other vehicles and convoy coordination in real traffic flow. This embodiment realizes multi-vehicle collaborative grouping training by establishing logical following relationships.

[0062] Specifically, the system features a virtual fleet training mode, allowing administrators to specify or the system to automatically generate a queue of training vehicles with logical following relationships. This logical following relationship does not require vehicles to strictly maintain a fixed geometric formation in physical space, but rather is based on a dependency at the digital trajectory level, giving the training mode greater flexibility. When a new training vehicle applies to join, the system provides two modes: a single-vehicle free training mode, or a group training mode that adds it to a designated virtual fleet.

[0063] In platoon training mode, the system dynamically assigns a platoon sequence number to each new vehicle. This sequence number not only identifies the vehicle's position within the platoon but also binds it to its following object in the logical chain. At this time, the digital standard training trajectory bound to the vehicle will no longer be generated solely based on a static map, but will be dynamically adjusted according to the real-time position and speed of the lead vehicle in the platoon to maintain platoon formation and a safe following distance.

[0064] As a specific implementation method, the calculation logic for dynamic adjustment is as follows: The system collects the pose data of the lead vehicle in real time. And based on the preset safe following distance (For example, set to 2 seconds) and minimum safe distance (For example, set to 5 meters), calculate the desired target position of the following vehicle. Desired position of the following vehicle. The lead car moves back a certain distance along the trajectory from its current position. Among them The system updates the target points of the following vehicle's trajectory in real time. It generates smooth control commands to guide the following vehicle to approach the target point.

[0065] It should be understood that if there are multiple following vehicles in a convoy, each following vehicle dynamically follows the vehicle in front of it, forming a cascading following chain. Furthermore, the system broadcasts the relative positions and statuses of vehicles within the convoy in real time, as part of the augmented reality guidance information for the participating vehicles. For example, drivers of following vehicles can see the virtual outline of the vehicle in front and real-time distance prompts through onboard displays or AR projections on their windshields, helping them maintain a safe following distance. Through this flexible grouping mechanism, the system can not only simulate convoy driving scenarios on real roads but also automatically adjust the grouping structure when vehicles dynamically enter or leave the training area, maintaining training order without manual intervention, greatly improving the intelligence level of the training system and the efficiency of site utilization.

[0066] Example 5:

[0067] Based on Example 1, this example elaborates in detail the specific composition of the high-precision field-end positioning and sensing subsystem and its data fusion mechanism with the multi-agent collaborative training management and dynamic broadcasting unit.

[0068] The system also includes a high-precision field-end positioning and sensing subsystem, comprising: multiple UWB positioning base stations deployed at key locations in the training field, used to communicate with UWB tags mounted on training vehicles to achieve centimeter-level real-time positioning of vehicles within the field. Specifically, UWB (Ultra-Wideband) technology, due to its extremely high temporal resolution, can provide centimeter-level ranging accuracy.

[0069] In terms of hardware deployment, at least four UWB base stations are typically installed on high poles at the four corners of the training ground and key intersections to provide signal coverage for the entire area. Vehicle-mounted UWB tags transmit pulse signals at a certain frequency (e.g., 10Hz). Each base station receives the signals and calculates the Time Difference of Arrival (TDOA) or Time of Arrival (TOA), using a triangulation algorithm to determine the vehicle's planar coordinates. It should be understood that although UWB positioning has high accuracy, its signal stability may be affected in environments with severe multipath effects (such as near metal containers or buildings), thus requiring auxiliary positioning methods.

[0070] This subsystem also includes roadside vision sensors deployed near lane lines to capture detailed behaviors such as vehicle crossing the lines and parking positions. These sensors are then fused with UWB positioning data to provide higher accuracy pose estimation. Roadside vision sensors typically use high-frame-rate industrial cameras, deployed at a height of 5-8 meters above the ground, to capture images of the lane line area from above. Through image processing algorithms, such as edge detection and Hough transform, the relative positional relationship between the vehicle's tires and the lane lines can be accurately identified, determining whether there is any line crossing. While this vision-based relative positioning method has limited coverage, it offers higher reliability than UWB in determining whether a vehicle has accurately entered a parking space or crossed the lines.

[0071] The multi-agent collaborative training management and dynamic broadcasting unit integrates the UWB positioning data and roadside visual data as redundant verification and optimization inputs for training the vehicle's real-time pose information, thereby improving the accuracy of trajectory tracking and collision detection. Specifically, the data fusion algorithm can employ extended Kalman filtering (EKF) or unscented Kalman filtering (UKF). In the algorithm model, UWB positioning data serves as the global position observation, providing absolute coordinates; vehicle-mounted GNSS / IMU data serves as the prediction value, providing high-frequency speed and attitude information; and roadside visual data serves as the local constraint observation, correcting the vehicle's lateral offset.

[0072] When a UWB signal abruptly changes, the system automatically reduces its weight in the observation noise matrix, relying on IMU calculations and visual data for a smooth transition. When the vehicle enters the coverage area of ​​the visual sensor, the system uses the lane distance parameters identified visually to laterally correct the UWB positioning results. This multi-source heterogeneous data fusion mechanism not only achieves centimeter-level positioning in all weather and all scenarios but also effectively solves the problem of single sensor failure in specific scenarios. It provides a solid data foundation for subsequent trajectory tracking error calculation and multi-vehicle conflict detection, ensuring the fairness of training evaluation and the safety of the training process.

[0073] Example 6:

[0074] Building upon Example 1, this example details the specific algorithmic logic for multi-dimensional comparison and risk assessment performed by the multi-modal acquisition and real-time evaluation feedback unit of driving behavior. This evaluation algorithm aims to transform abstract driving operations into quantifiable data indicators, thereby achieving precise closed-loop teaching.

[0075] First, in terms of trajectory tracking error, this unit calculates the root mean square error of the lateral deviation and heading deviation between the vehicle's actual trajectory and the digital standard training trajectory. Specifically, the lateral deviation is defined as the vertical distance from the vehicle's center of mass to the projection point of the digital standard training trajectory, and the heading deviation is defined as the angle between the vehicle's actual heading angle and the tangent direction of the standard trajectory.

[0076] Because deviations during driving are random, a simple arithmetic mean would lose its evaluative significance due to the cancellation of positive and negative values. Therefore, this embodiment uses the root mean square error (RMSE) as the evaluation index. Taking lateral deviation as an example, its calculation formula is as follows: ,in The number of sampling points. For the first The lateral deviation value at each sampling point. A higher RMSE value indicates a worse ability of the trainee to maintain lane position. The system sets grading thresholds; for example, an RMSE less than 0.2 meters is considered excellent, and a RMSE greater than 0.5 meters is considered unqualified.

[0077] Secondly, in terms of operational smoothness, this unit analyzes the power spectrum of the rate of change of steering wheel angle and the rate of change of pedal travel to evaluate the smoothness and predictability of operation. It should be understood that the operation of skilled drivers is usually predictable, and the control signals of the steering wheel and pedals are mainly low-frequency components; while novice drivers, due to tension or delayed reaction, often make high-frequency corrective actions (such as frequent fine-tuning of the steering wheel and brake nodding).

[0078] This embodiment uses Fast Fourier Transform (FFT) to convert the time-domain operating signal into the frequency-domain power spectral density (PSD). The system focuses on monitoring the proportion of energy in the high-frequency band above 1Hz. If the proportion of high-frequency energy in the total energy exceeds a preset threshold (e.g., 20%), the smoothness of operation is deemed unqualified, indicating that the driver has bad habits such as "rushing the steering wheel" or "sudden braking".

[0079] Secondly, regarding risk indicators, this unit combines parameters such as real-time vehicle speed, minimum distance to surrounding virtual or real obstacles, and TTC (Time to Collision) to calculate the comprehensive risk coefficient at the current moment. Among these, TTC is the core indicator for measuring collision risk, and its calculation formula is as follows: ,in This is the relative distance between the vehicle and the obstacle. This is the relative velocity. When the obstacle is a stationary object, This is the speed of the vehicle. Overall risk factor. The computational model can be represented as: ,in These are the weighting coefficients. This model comprehensively considers speed risk, distance risk, and collision time risk, and can more fully reflect the degree of danger in the current driving state.

[0080] Finally, based on the above multi-dimensional evaluation results, the real-time generated correction prompts will be generated in a targeted manner according to the main deviation dimensions, including virtual lane line guidance for correcting lateral deviation, "smooth steering" text for prompting overly aggressive operation, and a conspicuous red flashing border warning when the risk coefficient exceeds the threshold.

[0081] For example, when the system detects that the lateral deviation RMSE exceeds the standard, but the smoothness and risk coefficient are normal, it determines that the student's main problem is inaccurate path planning. The generated correction prompt is a bright green virtual lane line overlaid on the augmented reality screen to guide the student to correct their direction. When the system detects that the high-frequency component of the steering wheel is too high, the generated correction prompt is a voice announcement saying "Please operate the steering wheel smoothly." When the TTC falls below the safe threshold (e.g., 2 seconds), the system immediately generates a prominent flashing red border on the display interface, accompanied by a rapid beeping sound, forcibly reminding the driver to take evasive action. This targeted correction mechanism avoids the drawbacks of traditional instructors who "only criticize without guiding," achieving precise teaching.

[0082] Example 7:

[0083] Building upon Example 1, this example details the specific functional implementation of the training result quantitative evaluation and report generation unit. As the final link in the training loop, this unit aims to transform real-time data streams into highly readable and clearly instructive evaluation reports, providing data support for subsequent teaching improvements.

[0084] The system also includes a training result quantitative evaluation and report generation unit, used to summarize all evaluation data from the driving behavior multimodal acquisition and real-time evaluation feedback unit throughout the entire trajectory cycle after a training task is completed. Specifically, the system automatically triggers the settlement process when the vehicle leaves the training area or completes the predetermined route. The summarization process is not a simple data accumulation, but rather involves extracting statistical features from the time-series data generated in Example 6.

[0085] For example, for trajectory tracking error, the system extracts the mean, maximum, and variance of the lateral deviation RMSE throughout the entire process; for operation smoothness, the system calculates the cumulative proportion of high-frequency energy in the power spectrum of the steering wheel angle change rate; for risk indicators, the system statistically analyzes the cumulative duration and number of times the risk coefficient exceeds a preset threshold (e.g., 0.8). This raw data is stored in a structured manner as an objective basis for scoring, avoiding the subjectivity and bias of traditional manual evaluation.

[0086] This unit is used to calculate the driver's score and total score across multiple categories, including trajectory tracking accuracy, operational smoothness, rule compliance, and risk avoidance ability, based on a preset weighting system. It should be understood that the weighting system is not static but dynamically adjusted according to the teaching objectives of the training subjects.

[0087] For example, in the "unprotected left turn" scenario training, the focus is on assessing the trainee's judgment and risk avoidance capabilities in complex traffic flow. The weight of risk avoidance ability can be set at 40%, trajectory tracking accuracy at 30%, operational smoothness at 20%, and rule compliance at 10%. In the "reversing into a parking space" scenario, the focus is on assessing the control of the vehicle's position, and the weight of trajectory tracking accuracy is increased to 50%. The scoring algorithm uses piecewise linear mapping or Gaussian kernel function mapping to convert physical dimensions (such as meters and Hertz) into standard scores.

[0088] For example, a lateral deviation RMSE less than 0.1 meters earns full marks, a linear deduction occurs within the 0.1-0.3 meter range, and a deviation greater than 0.3 meters earns zero marks. The total score is calculated as follows: ,in For the first The weight of each item The score is assigned to this sub-item. This quantitative scoring mechanism ensures the fairness and traceability of the evaluation results.

[0089] This unit automatically generates structured training reports, including score summaries, key event replays, strengths and areas for improvement analysis, and supports trend analysis comparing with historical training records. The report generation module includes built-in natural language generation templates. The score summary section typically uses radar charts or bar graphs to visually display the trainee's performance across various ability dimensions.

[0090] The key event replay section extracts the three segments with the highest risk coefficient or the largest deviation during the training process, associates them with the augmented reality guidance information frames at that time and vehicle operation data, and generates short videos or trajectory animations that can be replayed for instructors and trainees to review and analyze. The strengths and areas for improvement analysis section is based on data mining logic. For example, the system identifies that the trainee's steering operation is highly smooth, demonstrating good predictive ability, but there are deficiencies in following distance control at complex intersections. The system will automatically generate a prompt to "strengthen defensive driving training".

[0091] In addition, the system stores the training data in the student profile database, calls up historical data to generate trend curves, and displays the changing trend of the student's "risk avoidance ability" score in the past ten training sessions, thereby verifying the teaching effect and realizing the visual management and scientific evaluation of the training process.

[0092] Example 8:

[0093] Building upon Example 1, this example details the specific structure of the immersive environment simulation execution unit and its linkage mechanism with the system control terminal. To further enhance the immersiveness and practicality of driving training, relying solely on visual augmented reality guidance is often insufficient. This example introduces physical environment simulation methods, enabling trainees to experience the harsh weather and complex lighting challenges of a real road environment even in a closed space.

[0094] The system also includes an immersive environment simulation execution unit, comprising: a controllable sprinkler system deployed on specific road sections to simulate low-adhesion road surfaces or visual interference scenarios such as rain, water splashing from vehicles ahead, etc. Specifically, the controllable sprinkler system is not a simple landscaping sprinkler head, but rather uses high-pressure micro-mist nozzles or fan-shaped nozzle arrays, deployed on roadside poles on both sides of specific road sections in the training area, typically at a height between 0.5 meters and 1 meter, to simulate the effect of water splashing from the road surface, or deployed on tall poles to simulate a rainy environment. This device is connected to a PLC controller via a solenoid valve, and integrated into the control network of a multi-agent collaborative training management and dynamic broadcasting unit.

[0095] When simulating low-adhesion road surfaces in rainy weather, the system controls the spray device to generate a fine water mist, reducing the road surface friction coefficient to 0.3-0.4, closely resembling the physical state of a real wet and slippery road surface. When simulating water splashing from a vehicle ahead that interferes with visibility, the system controls the side nozzles to spray water the instant the vehicle passes, simulating a sudden visual obstruction. This physical-level environmental simulation allows the driver to realistically feel the tendency of the vehicle to slip on a wet and slippery road surface and the operating pressure of the windshield wipers—a "road feel" that pure virtual simulation cannot replace.

[0096] This unit also includes intelligent light and shadow projection devices deployed along the roadside to simulate lighting changes such as glare from oncoming vehicle high beams and transitions between light and dark in tunnels. These intelligent light and shadow projection devices typically employ high-brightness LED strobe lights or programmable intelligent spotlights, deployed at simulated tunnel entrances and exits, the end of straight sections, or oncoming lanes on curves.

[0097] These devices boast millisecond-level response times, enabling precise control of light intensity and flicker frequency. For example, in simulating a nighttime oncoming traffic scenario, when the training vehicle reaches a specific trigger point, the oncoming intelligent light projection device instantly activates a high-intensity light mode, simulating the glare of high beams directly hitting the driver's retina, forcing the driver to avoid eye contact or operate the visor. In simulating tunnel entrance / exit scenarios, the device rapidly adjusts the light intensity in accordance with the vehicle's speed, simulating the "dark adaptation" process of the human eye when suddenly entering a dark environment from a bright one. Through this physical stimulation of light and shadow, the system effectively trains the driver's visual adaptation ability and psychological resilience under strong light interference.

[0098] The multi-agent collaborative training management and dynamic broadcasting unit can send trigger commands to the environment simulation execution unit when the vehicle reaches a specific location, based on scene tags in the digital standard training trajectory, thereby reproducing the real-world environmental challenges from the V2X data source within the physical training field. This process embodies the reverse control of the physical world by digital twin technology. In the V2X data processing stage described in Example 1, the system not only extracts the vehicle trajectory but also extracts environmental metadata (such as weather conditions and lighting conditions) at the time the trajectory occurs, and encapsulates it into scene tags (such as "rainy day" and "nighttime glare") and binds them to key nodes of the digital standard training trajectory.

[0099] As the training vehicle executes the trajectory in real time, the multi-agent collaborative training management and dynamic broadcasting unit continuously compares the vehicle's real-time pose with trajectory nodes. Once the vehicle enters a road section tagged with "rainy," the system immediately sends an activation command to the sprinkler controller in that section; when the vehicle approaches a coordinate point tagged with "glare," the system sends a trigger pulse to the corresponding lighting device. This mechanism, based on precise spatiotemporal location triggering, ensures perfect synchronization between the physical environment simulation and the virtual scene scenario, achieving a truly realistic simulation training effect of "what you see is what you feel," and greatly enhancing the driver's practical experience in dealing with complex weather conditions.

[0100] Example 9:

[0101] Building upon Example 1, this example details the specific hardware configuration and implementation of the display device in the distributed augmented reality guidance display array. The display device in the distributed augmented reality guidance display array is either a transparent LED screen or a laser projection device. These two configurations are designed to meet different visual presentation needs and site conditions, aiming to maximize the immersiveness and practicality of augmented reality guidance information.

[0102] When using transparent LED screens, they are installed above the lane, and the displayed augmented reality guidance information frames are blended and overlaid with the real road background seen by the driver. Specifically, the transparent LED screens use high-transmittance patch technology, with a transmittance typically set between 70% and 85%. This ensures both high brightness of the virtual image and allows the driver to clearly observe the real road environment, traffic lights, and distant landmarks behind the vehicle.

[0103] Regarding installation height, it is generally set at 4.5 to 5.5 meters above the lane. This height avoids collisions with large training vehicles while remaining within the optimal field of vision for the driver, either at eye level or slightly upward, reducing visual fatigue caused by changing gaze. The advantage of transparent LED screens lies in their ability to present three-dimensional virtual objects with depth information. For example, when the system needs to simulate a "virtual vehicle cut-in" scenario, the virtual vehicle image rendered on the transparent LED screen will visually blend perfectly with the real road background behind it. The driver can intuitively perceive the virtual vehicle's positional relationship in the real environment, thus generating a strong "virtual-real symbiosis" visual experience, greatly enhancing the realism of the training.

[0104] When using laser projection equipment, guidance information is projected directly onto the training ground surface in front of the vehicle, creating virtual directional signs that blend seamlessly with the real environment. Specifically, laser projection equipment is typically deployed on roadside poles or gantries, projecting onto the road surface from a top-down angle. The projection angle requires strict calibration, usually between 30 and 60 degrees to the road surface normal, to minimize obstruction of the projection light path by vehicles and to ensure that the geometry of the virtual signage is not severely distorted in the driver's field of vision.

[0105] Regarding the road surface color rendering effect, the equipment uses a high-brightness laser light source, typically requiring a brightness of over 5000 lumens to resist interference from ambient daylight and ensure sufficient contrast and saturation for virtual lane lines, directional arrows, and other planar guidance information on asphalt or concrete pavements. The advantages of laser projection equipment are that it eliminates the need for high-altitude screens, resulting in relatively low construction costs. Furthermore, the projected virtual lane lines, driving trajectory strips, and other planar information are visually highly consistent with real road markings, eliminating the need for drivers to switch their viewing distance vertically and aligning more with the natural habit of looking down to observe the road while driving.

[0106] It should be understood that the two forms mentioned above each have their own focus and are suitable for different training scenarios. Transparent LED screens are more suitable for displaying three-dimensional virtual objects with high information (such as virtual vehicles, pedestrians, and obstacles), and can provide richer spatial location cues; while laser projection equipment is more suitable for displaying planar guidance information (such as dynamic lane lines, speed limit signs, and driving trajectory guidance strips), and has better visual effects at night or in low-light environments.

[0107] In actual deployment, training grounds can choose to deploy one of the forms individually or adopt a hybrid deployment mode, depending on the specific budget and subject settings. For example, transparent LED screens can be installed above key intersections to display complex interactive vehicles, and laser projection can be used to display dynamic guide lines in straight road areas, thereby achieving the optimal balance between cost and effect.

[0108] Example 10:

[0109] like Figure 1 As shown, this embodiment provides a driving training method based on V2X data, such as... Figure 2 As shown, this method is applied to the driving training system described in the foregoing embodiments. This method constructs a complete closed loop from data preparation, real-time scheduling, visual presentation to evaluation feedback in terms of temporal logic, specifically including the following steps: Step S101, Digital Twin Trajectory Construction Step: Process V2X historical data, mine typical high-risk interaction scenarios, and generate a digital standard training trajectory mapped to the physical training field and a synchronized augmented reality guidance information sequence.

[0110] Specifically, this step is the "data foundation" of the entire training method. The system first accesses the city's V2X historical database, performing cleaning and denoising operations to remove abnormal trajectory points caused by signal packet loss. Then, the system uses a key scene mining algorithm to perform differential processing on the cleaned full trajectory data, calculating dynamic characteristics such as jerkiness and lateral offset rate, and filtering out high-risk interaction scene segments containing cut-in and emergency braking. Based on this, the system registers these real-world scene segments with a high-precision 3D point cloud map of a closed training field, and through scale transformation and coordinate transformation, generates a digital standard training trajectory that can be executed within the physical venue.

[0111] Meanwhile, the system generates corresponding augmented reality guidance information frames for each key decision point in the trajectory. These information frames contain the position, posture, and expected driving operation guidance of the virtual traffic participants, thus forming a complete "script" that provides a digital blueprint for subsequent real vehicle training.

[0112] Step S102, Multi-agent collaborative management steps: Dynamically allocate and bind training trajectories to multiple training vehicles, predict trajectory and detect conflicts in real time, dynamically adjust the plan to ensure the safety of collaborative training, and calculate and publish guidance information.

[0113] This step is the "central scheduling" stage of the training process. When a training vehicle enters the training area, the system dynamically assigns a digital standard training trajectory according to its training plan and establishes a binding relationship between the vehicle and its execution trajectory. During training, the system receives fused positioning data reported by each vehicle in real time and predicts the movement trajectory of each vehicle in the next 3-5 seconds at a high frequency (e.g., 50ms period). The system constructs a three-dimensional dynamic safety envelope for each vehicle and detects whether the safety envelopes of any two vehicles intersect in the spatiotemporal domain.

[0114] Once a potential conflict is detected, the system immediately triggers dynamic replanning logic to locally correct the trajectory of the vehicle obligated to yield, such as by inserting deceleration points or adjusting the path, thereby eliminating the collision risk at the physical level. Simultaneously, based on the real-time pose and bound trajectory of each vehicle, the system calculates the augmented reality guidance information to be presented in the next moment and multicasts it to the corresponding display devices via a low-latency network, ensuring the real-time nature and accuracy of the guidance information.

[0115] Step S103, Augmented Reality Guidance Step: Augmented reality guidance information is dynamically presented based on the real-time location of the vehicle through a distributed display array, providing immersive visual guidance.

[0116] This step achieves "virtual-real fusion" between the virtual scene and the physical environment. Distributed augmented reality guidance display arrays (such as transparent LED screens or laser projection devices) deployed at key locations in the training field continuously monitor the guidance information broadcast by the system. When a vehicle travels to a specific area, the corresponding display device renders virtual elements such as vehicles, pedestrians, or lane lines in real time based on the received information frames.

[0117] These virtual elements are visually superimposed on the real road background that the driver sees through the car window, allowing the driver to perceive the complex traffic flow of the real road environment in a closed space, thus providing highly immersive visual guidance and effectively solving the shortcomings of the single training scenario in traditional training.

[0118] Step S104, Closed-loop evaluation and feedback step: Multimodal acquisition of driving behavior, real-time comparison and evaluation with digital twin standards, and real-time integration of correction prompts into guidance information to achieve closed-loop training of "guidance-operation-evaluation-correction".

[0119] This step establishes an efficient "teaching interaction" mechanism. The system collects multimodal operational data from the driver in real time, including steering wheel angle, pedal travel, and gaze area, using onboard sensors, and aligns this data spatiotemporally with a pre-set expected operational benchmark model in a digital standard training trajectory. The system calculates multi-dimensional evaluation parameters such as trajectory tracking error, operational smoothness, and risk indicators.

[0120] When a significant deviation is detected between the trainee's actions and the standard model, such as delayed steering or excessive braking, the system generates targeted corrective prompts in real time. These prompts are not simply voice announcements, but are overlaid in real-time into the augmented reality guidance information frame to be broadcast. For example, a corrected virtual trajectory line is overlaid on the display screen to guide the trainee back to the correct route, thus achieving an instantaneous closed loop from guidance to operation, and then to evaluation and correction, greatly improving training efficiency.

[0121] Step S105, Environment Simulation and Report Generation Step: Trigger the physical environment simulation device to reproduce the complex scene, and generate a quantitative evaluation report and trend analysis after training.

[0122] This step further enhances the "realism" and "scientific rigor" of the training. During training, the system sends trigger commands to the physical environment simulation device when the vehicle reaches a specific location, based on scene tags in the digital standard training trajectory. For example, it controls the sprinkler system to simulate a slippery road surface in rain, or controls the lighting device to simulate a glare scene at night, allowing trainees to experience real-world environmental challenges on a physical level.

[0123] After training, the system summarizes the evaluation data from the entire process, calculates scores for each ability dimension based on preset weights, and automatically generates a structured training report. The report not only includes a score summary but also provides trainees with objective and quantifiable evidence of skill improvement through key event replays and historical trend comparisons, completing a full methodological loop from data input to effect evaluation.

[0124] Example 11:

[0125] To more intuitively verify the effectiveness and safety of the driving training system provided by this invention in handling complex dynamic interactive scenarios, this embodiment selects the "unprotected left turn and cut-in composite scenario" as a specific application case for detailed explanation. It should be understood that this scenario is merely an example, and this invention is also applicable to training in other high-risk scenarios such as emergency avoidance and unexpected pedestrian collisions.

[0126] Step S1101, Digital Twin Scenario Construction Phase. The system first retrieves and extracts a real high-risk traffic accident segment from the city's V2X historical database. This segment records a near-miss incident at a city intersection during the morning rush hour, where a vehicle going straight through the intersection collides with a vehicle turning left. The V2X data deep processing and digital trajectory twin unit cleans and re-identifies the raw data, parsing out the trajectory features of the key interacting vehicles.

[0127] Specifically, the system detected that the peak speed of the left-turning vehicle in this segment reached 4.5 m / s. 3Furthermore, the minimum spatial distance between the real intersection and the straight-going vehicle is only 2.3 meters, meeting the criteria for a high-risk interaction scenario. Subsequently, the system registers the geometric features of the real intersection with the physical space of the closed training ground. Through a scaling algorithm, the original 30-meter-wide real intersection is mapped to a 15-meter-wide simulated intersection within the training ground, generating a digital standard training trajectory. In this trajectory, training vehicle A is assigned to perform an unprotected left turn, while virtual vehicle B is set to perform a straight-going cut-in task. Its entry timing is precisely set 2 seconds before vehicle A reaches the center point of the intersection to simulate the most dangerous game scenario.

[0128] Step S1102, Training Deployment and Initialization Phase. Before the actual training begins, the system deploys a transparent LED screen above the simulated intersection as the core display device of the distributed augmented reality guidance display array. Training vehicle A enters the area, and the multi-agent collaborative training management and dynamic broadcasting unit binds it to the aforementioned generated digital standard training trajectory. At this time, the system uses the fused positioning data from UWB base stations and roadside visual sensors to lock vehicle A's initial position in the left-turn waiting area in real time, with centimeter-level accuracy.

[0129] Step S1103, Dynamic Interaction and Conflict Detection Phase. When vehicle A enters the intersection to make a left turn, the multi-agent collaborative training management and dynamic broadcasting unit predicts vehicle A's trajectory over the next 3 seconds based on a kinematic model and generates a three-dimensional dynamic safety envelope. Simultaneously, the system controls a transparent LED screen to render an image of virtual vehicle B in real time. Virtual vehicle B rapidly approaches from the oncoming lane at a virtual speed of 60 km / h. The system continuously monitors the spatiotemporal relationship between vehicle A's safety envelope and virtual vehicle B's predicted trajectory.

[0130] In the standard training mode set in this case, the behavioral logic of virtual vehicle B is predetermined, aiming to train the driver of vehicle A's ability to predict and avoid collisions. However, if the system detects that vehicle A's safety envelope intersects with the trajectory of virtual vehicle B due to operational errors, the system will immediately determine it as a potential conflict and trigger the collision avoidance warning mechanism.

[0131] Step S1104, Operation Acquisition and Real-time Evaluation Phase. As vehicle A approaches the conflict point, the driving behavior multimodal acquisition and real-time evaluation feedback unit collects data such as the driver's steering wheel angle and brake pedal travel at high frequency through onboard sensors. Assume that the driver, upon observing the virtual vehicle B cutting in, brakes suddenly due to nervousness, causing the brake pedal travel rate to instantaneously exceed a preset threshold.

[0132] The system compared the real-time collected multimodal operation data with the expected operation baseline model and found that the sudden braking operation caused significant longitudinal acceleration fluctuations, and that the high-frequency energy accounted for an excessively high proportion in the power spectrum analysis of the operation smoothness dimension. Based on this, the system calculated that the comprehensive risk coefficient at the current moment rapidly increased to 0.85 (exceeding the safety threshold of 0.7).

[0133] Step S1105, Closed-loop Feedback and Correction Phase. Based on the above evaluation results, the system immediately generates a correction prompt. On one hand, the system overlays a conspicuous flashing red border warning onto the transparent LED screen display for the driver of vehicle A, accompanied by dynamic guidance from virtual lane lines, prompting the driver to correct the steering angle to avoid the driving path of virtual vehicle B. On the other hand, the system injects the correction prompt "Braking too suddenly, anticipatory deceleration recommended" into the augmented reality guidance information frame, achieving immediate feedback.

[0134] After vehicle A safely passed the intersection, the training result quantitative evaluation and report generation unit summarized the data from the entire process and generated a structured report. The report clearly pointed out that although the driver ultimately avoided a collision in this composite scenario training, there were deficiencies in risk prediction and operational smoothness, and retrieved key event replay clips for debriefing and analysis.

[0135] As demonstrated by the above application examples, this invention can safely and controllably reproduce complex real-world traffic conflict scenarios within a closed environment. The system not only provides highly immersive visual guidance but also effectively trains drivers' stress responses and operational skills under extreme conditions through real-time conflict detection and closed-loop evaluation feedback mechanisms, validating the practical application value of the technical solution.

[0136] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention, such as adjustments to the threshold in the V2X data mining algorithm, equivalent replacements of the display device hardware form, or conventional optimizations of the multi-agent collaborative control logic, should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0137] It should be understood that although this specification describes various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0138] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A driving training system based on V2X data, characterized in that, Deployed within a closed training ground equipped with multiple simulated lanes and environmental simulation facilities, the system includes: The V2X data deep processing and digital trajectory twin unit is used to access the urban road network V2X historical database, clean, denoise and re-identify vehicle trajectories of multi-source heterogeneous traffic flow data, and decouple to generate high-confidence single-vehicle continuous spatiotemporal trajectories; it is used to automatically extract complex interactive scene segments including cut-in, emergency braking and unprotected left turns from the single-vehicle spatiotemporal trajectory based on driving behavior dynamics model and key scene mining algorithm. This is used to register and scale the interactive scene fragments with the high-precision 3D point cloud map of the training field, generate a digital standard training trajectory that is completely mapped to the physical training field, and synchronously generate augmented reality guidance information frames containing the relative positions of multiple vehicles, traffic signal status, and expected operation guidance for each key decision point of the trajectory. The multi-agent collaborative training management and dynamic broadcasting unit is used to maintain the digital standard training trajectory library and the corresponding augmented reality guidance information frame sequence; it is used to receive in real time the precise pose, velocity and control status reported by each training vehicle in the field, which integrates vehicle-mounted GNSS / IMU and UWB positioning data; Used to dynamically bind and update the currently executed training trajectory for each training vehicle; used to predict the future short-term trajectory of each vehicle based on the kinematic model, and to detect potential conflicts between multiple vehicles in real time based on the predicted trajectory and the preset collision avoidance safety envelope; when a potential conflict is detected, to dynamically adjust the subsequent trajectory point sequence or guidance information of the affected vehicles, and to re-plan the broadcast timing; used to calculate the augmented reality guidance information frame to be presented in the next moment and its corresponding target display device identifier based on the real-time pose of the vehicle, the bound trajectory and the collision avoidance decision, and to multicast it through a low-latency communication network; The distributed augmented reality guidance display array consists of multiple display devices with independent identifiers, which are deployed above or to the side of the lane according to the key location topology in the 3D point cloud map of the training field. Each display device is used to listen to the multicast information of the dynamic broadcast unit. When the target identifier in the multicast information matches the identifier of the device, the augmented reality guidance information frame is rendered and displayed in real time to provide the driver with highly immersive visual guidance. The multimodal acquisition and real-time evaluation feedback unit for driving behavior is used to acquire multimodal operation data of the driver through high-precision sensors deployed on the training vehicle, including steering wheel angle / torque, pedal travel / rate, gear signal, and gaze area; and to establish the expected operation benchmark model for each key decision point in the digital standard training trajectory. This is used to perform spatiotemporal alignment and multidimensional comparison of real-time collected multimodal operation data with the expected operation benchmark model of the current decision point, and to calculate trajectory tracking error, operation smoothness and risk indicators; based on the comparison and calculation results, it is used to generate evaluation conclusions and correction prompts in real time, and to inject the correction prompts into the augmented reality guidance information frame to be broadcast by the dynamic broadcast unit in the form of an overlay layer, forming a closed-loop feedback.

2. The driving training system according to claim 1, characterized in that, The V2X data deep processing and digital trajectory twin unit extracts complex interaction scene fragments based on key scene mining algorithms, including: Differentiate the spatiotemporal trajectory of a single vehicle to calculate the time series of its acceleration, jerk, and lateral position offset rate; Set the agility threshold, lateral offset rate threshold, and minimum spatial distance threshold from surrounding vehicles; When trajectory point data simultaneously meets the conditions of swiftness exceeding the threshold, lateral offset exceeding the threshold, and spatiotemporal distance from surrounding vehicles below the threshold, the trajectory segment is marked as a high-risk interaction scenario. Clustering algorithms are used to classify all high-risk interaction scenarios, and the most representative segments are selected from each category according to the teaching syllabus of the training system to form a standard training scenario library.

3. The driving training system according to claim 1 or 2, characterized in that, The multi-agent collaborative training management and dynamic broadcasting unit predicts conflicts and dynamically adjusts them based on kinematic models, including: Using the current state of each vehicle as the initial value, a simplified single-vehicle kinematics model is used to predict the trajectory within the next 3 to 5 seconds, forming a cluster of predicted trajectories. Define a three-dimensional dynamic safety envelope for each vehicle that includes its dimensions and safety margin, and expand it along its predicted trajectory cluster; Real-time detection of whether the expanded safety envelopes of any two vehicles intersect in the spatiotemporal domain; If an intersection is detected, a potential conflict is determined. The digital standard training trajectory bound to the following vehicle or the vehicle that is logically obligated to yield is immediately replanned locally. By inserting deceleration points, modifying waypoints, or increasing waiting time, a conflict-free corrected trajectory segment is generated, and the subsequent guidance information frame sequence is updated.

4. The driving training system according to claim 1, characterized in that, The multi-agent collaborative training management and dynamic broadcasting unit is also used to realize flexible grouping and dynamic entry of training vehicles, including: The virtual fleet training mode allows administrators to specify or the system to automatically generate a group of training vehicles with logical following relationships. Two modes are provided for newly applied training vehicles: single-vehicle free training mode, or group training mode that joins a designated virtual fleet; In the group training mode, a new vehicle is dynamically assigned a platoon sequence number, and its bound digital standard training trajectory will be dynamically adjusted according to the real-time position and speed of the lead vehicle in order to maintain the platoon formation and safe following distance. The relative positions and statuses of vehicles within the convoy are broadcast in real time as part of the augmented reality guidance information for team members' vehicles to reference.

5. The driving training system according to claim 4, characterized in that, The system also includes a high-precision field-end positioning and sensing subsystem, which includes: Multiple UWB positioning base stations deployed at key locations in the training field are used to communicate with the UWB tags on the training vehicles to achieve centimeter-level real-time positioning of the vehicles within the field. Roadside vision sensors deployed near lane lines are used to capture detailed behaviors such as vehicle crossing the lines and parking positions, and are fused with UWB positioning data to provide higher accuracy pose estimation. The multi-agent collaborative training management and dynamic broadcasting unit integrates the UWB positioning data and roadside visual data as redundant verification and optimization input for the real-time pose information of the training vehicle, thereby improving the accuracy of trajectory tracking and conflict detection.

6. The driving training system according to claim 1, characterized in that, The driving behavior multimodal acquisition and real-time evaluation feedback unit performs multidimensional comparison and risk assessment, including: Trajectory tracking error dimension: Calculate the root mean square error of lateral deviation and heading deviation between the actual vehicle trajectory and the digital standard training trajectory; Operational smoothness dimension: Analyze the power spectrum of the rate of change of steering wheel angle and the rate of change of pedal travel to evaluate the smoothness and predictability of operation; Risk indicator dimensions: The comprehensive risk coefficient at the current moment is calculated by combining parameters such as real-time vehicle speed, minimum distance to surrounding virtual or real obstacles, and TTC. The real-time generated correction prompts will be generated in a targeted manner based on the main deviation dimensions, including virtual lane line guidance for correcting lateral deviations, "smooth turn" text for indicating excessive operation, and a striking red flashing border warning when the risk factor exceeds the threshold.

7. The driving training system according to claim 6, characterized in that, The system also includes a training result quantitative evaluation and report generation unit, used for: After a training task is completed, all evaluation data from the driving behavior multimodal acquisition and real-time evaluation feedback unit during the entire trajectory cycle are summarized. Based on a preset weighting system, the driver's scores and total score are calculated on multiple sub-items such as trajectory tracking accuracy, operational smoothness, rule compliance, and risk avoidance ability. It automatically generates structured training reports, including score summaries, key event replays, strengths and areas for improvement analysis, and supports trend analysis by comparing with historical training records.

8. The driving training system according to claim 1, characterized in that, The system also includes an immersive environment simulation execution unit, which comprises: Controllable sprinkler systems deployed on specific road sections are used to simulate scenarios such as rain, water splashing from vehicles ahead, low-adhesion road surfaces, or visual interference. Intelligent light and shadow projection devices deployed on the roadside are used to simulate light changes such as the glare of oncoming vehicles' high beams and the transition between light and dark in tunnels. The multi-agent collaborative training management and dynamic broadcasting unit can send a trigger command to the environment simulation execution unit when the vehicle reaches a specific location based on the scene labels in the digital standard training trajectory, thereby reproducing the real environmental challenges in the V2X data source within the physical training field.

9. The driving training system according to claim 1, characterized in that, The display device in the distributed augmented reality guided display array is a transparent LED screen or a laser projection device; When a transparent LED screen is used, it is installed above the lane, and the displayed augmented reality guidance information frames can be blended and overlaid with the real road background seen by the driver. When using laser projection equipment, guidance information is projected directly onto the training ground road surface in front of the vehicle, forming a virtual guidance sign that is integrated with the real environment.

10. A driving training method based on V2X data, applied to the system as described in any one of claims 1-9, characterized in that, The method includes: The steps for constructing a digital twin trajectory are as follows: process historical V2X data, explore typical high-risk interaction scenarios, and generate a digital standard training trajectory mapped to the physical training field, along with a synchronized augmented reality guidance information sequence. Multi-agent collaborative management steps: dynamically allocate and bind training trajectories to multiple training vehicles, predict trajectory and detect conflicts in real time, dynamically adjust the plan to ensure the safety of collaborative training, and calculate and publish guidance information; Augmented reality guidance steps: Augmented reality guidance information is dynamically presented based on the vehicle's real-time location through a distributed display array, providing immersive visual guidance; Closed-loop evaluation and feedback steps: Multimodal collection of driving behavior, real-time comparison and evaluation with digital twin standards, and real-time integration of correction prompts into guidance information to achieve closed-loop training of "guidance-operation-evaluation-correction"; The physical environment simulation device is triggered to reproduce complex scenarios, and a quantitative evaluation report and trend analysis are generated after training.

Images