Digital twin-driven intelligent assisted driving guidance system and method

Abstract

The present invention discloses an intelligent assisted driving guidance system driven by a digital twin designed by the present invention, which is characterized in that: it includes a physical traffic entity layer, a digital traffic twin layer, a connection interaction layer, a traffic data center layer and an assisted driving guidance service layer; the present invention constructs a digital model that maps physical traffic entities and their behaviors and rules in a multi-dimensional, full-factor and high-fidelity manner, and uses a two-way feedback mechanism to realize information sharing among the physical traffic entity layer, the digital traffic twin layer and the assisted driving guidance service layer; it provides a number of intelligent assisted guidance services including vehicle status monitoring services, weather services, event guidance services, lane keeping services, blind spot monitoring services, and forward collision warning services, so that drivers can use the twin world to gain insight into the real world, increase the driver's understanding and response ability to the driving environment, realize human-machine co-driving, and provide safe service guidance for the driver.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and specifically to an intelligent assisted driving guidance system and method driven by a digital twin. Background Art

[0002] While driving, drivers rely on vision for over 80% of their information. However, blind spots and limited hearing often affect their judgment, leading to accidents and, in severe cases, even fatalities. Existing driving guidance systems, based on digital maps, primarily provide route-level guidance, determining specific driving routes through navigation maps. These systems only provide rough, point-to-point guidance, taking into account transportation methods, route distances, traffic conditions, and locations along the way. However, they lack effective guidance and assurance for safe driving.

[0003] Existing intelligent assisted driving guidance systems are primarily based on GPS positioning and map navigation. They only provide rough point-to-point road guidance, taking into account transportation methods, route distances, traffic conditions, and locations along the way. However, they lack effective guidance and guarantees for safe driving.

[0004] For example, the Chinese patent application number 201610972375 proposes a vehicle guidance block acquisition method and device as well as an autonomous driving method and system. It utilizes the advantages of low-precision maps and maps, combines multiple research methods, and plans paths so as to select a suitable driving path in autonomous driving. It only provides rough point-to-point road-level guidance, and does not take into account the impact of road conditions, environment, and traffic status information on the driving process, and cannot provide real-time and safe driving guidance services.

[0005] For example, the Chinese patent application number 201910018083 proposes a visibility-based obstacle avoidance driving guidance system and its guidance method. The driving information of vehicles on the lane is collected through the roadside unit, the meteorological visibility distance is detected through the visibility sensor, the object is detected and the motion state of the object is obtained through the microwave sensor, and the central processing unit performs calculations, processing, and judgment, and outputs instructions to the on-board display to meet the vehicle's obstacle avoidance requirements. The system mainly relies on on-board sensors, roadside sensors, and cloud platforms to obtain information, and only provides early warnings and projection displays for obstacle detection and avoidance. It does not take into account the impact of safety hazards other than obstacles on the driver. The information used and the functions provided are relatively simple.

[0006] Existing technologies lack comprehensive digital twin models that can simulate physical traffic entities, their behaviors, and rules with high fidelity, and limit the ability to provide personalized and customized guidance services; these shortcomings limit the ability of existing technologies to provide effective, safe, and convenient intelligent assisted driving services in complex, dynamic, and changing traffic environments. Summary of the Invention

[0007] The purpose of the present invention is to provide an intelligent assisted driving guidance system and method driven by digital twins. Based on digital twin drive, the present invention constructs a high-precision, high-reliability, and high-real-time traffic information perception and fusion mechanism to achieve all-round perception of road conditions, environment, and traffic status; the present invention constructs a multi-dimensional, full-factor, high-fidelity digital model that maps physical traffic entities and their behaviors and rules, and uses a two-way feedback mechanism to realize information sharing between the physical traffic entity layer, the digital traffic twin layer, and the assisted driving guidance service layer.

[0008] To achieve this goal, the digital twin-driven intelligent assisted driving guidance system designed by the present invention is characterized by comprising a physical traffic entity layer, a digital traffic twin layer, a connection interaction layer, a traffic data center layer, and an assisted driving guidance service layer;

[0009] The physical traffic entity layer includes objective physical entities involved in the real traffic environment and related data acquisition and perception equipment, which are used to collect and perceive perception data of the driving environment;

[0010] The traffic data center layer is used to receive, store and process digital twin data to drive the synchronous operation of the data-driven physical traffic entity layer, the digital traffic twin layer and the assisted driving guidance service layer. The digital twin data includes driving environment perception data, virtual model simulation data and driving guidance service operation data;

[0011] The connection interaction layer is used to forward the digital twin data of the traffic data center layer, and establish a two-way connection between the physical traffic entity layer, the digital traffic twin layer and the assisted driving guidance service layer through data synchronization and transmission;

[0012] The digital traffic twin layer is a digital mirror of the physical entities in the physical traffic entity layer. The digital traffic twin layer maps the physical entities in the physical traffic entity layer to obtain a virtual model. The virtual model includes a visual three-dimensional model, a physical property simulation model, a behavior model, and a rule model. Driven by the digital twin data in the traffic data center layer forwarded by the connection interaction layer, the virtual model reflects the behavior and status of the physical entities in the physical traffic entity layer in real time, thereby realizing the simulation of the physical entities in the physical traffic entity layer and synchronizing the virtual model simulation data generated by the simulation to the traffic data center layer through the connection interaction layer;

[0013] The assisted driving guidance service layer is a collection of assisted driving guidance services. The assisted driving guidance service layer uses the driving environment perception data and virtual model simulation data in the digital twin data forwarded by the connection interaction layer to provide assisted driving guidance services.

[0014] A digital twin-driven intelligent assisted driving guidance method, characterized in that it includes the following steps:

[0015] Step 1: Download map data from the data collection and perception devices at the physical traffic entity layer to the traffic data center. This map data is used to describe the topology, geometry, and logical attributes of the road network. The map data is processed in the traffic data center and parsed using the relevant parser. Through the interfaces and methods provided by the parser, the various elements of the map data are accessed and converted into a data structure that can be processed by the computer.

[0016] Step 2: For each parsed road data, discretize it to different degrees according to the type of road centerline in the traffic data center layer;

[0017] Step 3: The traffic data center layer forwards this discretized data to the digital traffic twin layer through the connection interaction layer. In the digital traffic twin layer, the vehicle's posture information is used to simulate the vehicle's position in the virtual model. Then, based on the vehicle's position in the digital traffic twin layer, a neighboring point matching algorithm is used to calculate the closest point to the road centerline in real time and find the most suitable road. This process uses a spatial index structure, the KD tree (k-dimensional tree), to retrieve road data from the map. The discretized points on the centerline of each road in the map are used as data points in the KD tree. The KD tree's nearest neighbor search algorithm is used to quickly find the nearest road data point and match it to the road to which it belongs.

[0018] Step 4: For the matched nearest point and the road it belongs to, the digital traffic twin layer calculates the difference between its coordinates and direction angles and those of the vehicle; based on these differences, it determines whether lane departure has occurred. If lane departure has occurred, the assisted driving guidance service layer provides the driver with corresponding warnings based on the degree and type of lane departure.

[0019] Beneficial effects of the present invention:

[0020] Based on digital twin drive, the present invention constructs a high-precision, high-reliability, and high-real-time traffic information perception and fusion mechanism to achieve all-round perception of road conditions, environment, and traffic status; constructs a multi-dimensional, full-factor, and high-fidelity digital model that maps physical traffic entities and their behaviors and rules, and uses a two-way feedback mechanism to achieve information sharing between the physical traffic entity layer, the digital traffic twin layer, and the assisted driving guidance service layer; provides a number of intelligent assisted guidance services including vehicle status monitoring services, weather services, event guidance services, lane keeping services, blind spot monitoring services, and forward collision warning services, so that drivers can use the twin world to gain insight into the real world, increase their understanding and response capabilities to the driving environment, realize human-machine co-driving, and provide drivers with safe service guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural schematic diagram of the present invention;

[0022] Figure 2 Flowchart for lane keeping services;

[0023] Figure 3 A schematic diagram of a first-person perspective in a vehicle blind spot monitoring service;

[0024] Figure 4 A schematic diagram of a third-person perspective in a vehicle blind spot monitoring service;

[0025] Figure 5 A schematic diagram of a bird's-eye view in a vehicle blind spot monitoring service;

[0026] Figure 6 A schematic diagram of the Eagle Eye (map) perspective in the vehicle blind spot monitoring service; DETAILED DESCRIPTION

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0028] like Figure 1 The digital twin-driven intelligent assisted driving guidance system is characterized by comprising a physical traffic entity layer, a digital traffic twin layer, a connection interaction layer, a traffic data center layer, and an assisted driving guidance service layer;

[0029] The physical traffic entity layer includes objective physical entities involved in the real traffic environment and related data acquisition and perception equipment, which are used to collect and perceive perception data of the driving environment;

[0030] The traffic data center layer is used to receive, store and process digital twin data to drive the synchronous operation of the data-driven physical traffic entity layer, the digital traffic twin layer and the assisted driving guidance service layer. The digital twin data includes driving environment perception data, virtual model simulation data and driving guidance service operation data;

[0031] The connection interaction layer is used to forward the digital twin data of the traffic data center layer, and establish a two-way connection between the physical traffic entity layer, the digital traffic twin layer and the assisted driving guidance service layer through data synchronization and transmission;

[0032] The digital traffic twin layer is a digital mirror of the physical entities in the physical traffic entity layer. The digital traffic twin layer performs multi-dimensional, full-factor, high-fidelity mapping of the physical entities in the physical traffic entity layer to obtain a virtual model. The virtual model includes a visual three-dimensional model, a physical property simulation model, a behavior model and a rule model. The virtual model comprehensively models multiple aspects of the physical entity. After mapping, the digital traffic twin layer can obtain a virtual model that highly restores the physical entity, which is used to simulate, predict and optimize the operating status of the traffic system, thereby providing support for fields such as traffic management and assisted driving. Driven by the digital twin data in the traffic data center layer forwarded by the connection interaction layer, the virtual model reflects the behavior and status of the physical entities in the physical traffic entity layer in real time, thereby achieving The physical entities in the physical traffic entity layer are simulated and the virtual model simulation data generated by the simulation is synchronized to the traffic data center layer through the connection interaction layer. The virtual model is used in the digital traffic twin layer to establish a mirror image of the real traffic environment and scene, model and simulate physical entities such as vehicles, roads, traffic signs, traffic lights, and visualize and analyze the perception data of the driving environment; the real physical traffic perception information is collected through sensors and other equipment and synchronized to the virtual model to update the status of the virtual model in real time, and the information from the physical traffic entity layer is mapped to the visual three-dimensional model, physical property simulation model, behavior model and rule model of the digital traffic twin layer, and the real traffic environment is simulated in the model to simulate and analyze various scenarios.

[0033] The assisted driving guidance service layer is a collection of assisted driving guidance services. The assisted driving guidance service layer uses the driving environment perception data and virtual model simulation data in the digital twin data forwarded by the connection interaction layer to provide assisted driving guidance services.

[0034] In the above technical solution, the physical entities in the physical traffic entity layer are the basic objects of the virtual model in the digital twin assisted driving system, including personnel, roads, roadside facilities, vehicles and sensors. The data acquisition and perception equipment in the physical traffic entity layer includes vehicle-mounted perception equipment, roadside perception equipment, cloud platform and map. The perception data of the driving environment are obtained through these data acquisition and perception equipment; the perception data of the driving environment includes the vehicle's posture information, status information and attribute information, road condition information and environmental condition information, wherein the vehicle's posture information is the position direction and angle information of the vehicle in three-dimensional space, and the vehicle's posture information includes longitude, latitude, altitude, pitch angle, roll angle and heading information, which is obtained by the vehicle The vehicle status information is obtained by the sensing device and the roadside sensing device; the vehicle status information is the vehicle's operating status and performance parameter information, and the vehicle status information includes engine speed, turn signal status, accelerator pedal position, brake pedal position, steering wheel angle and turn signal status, which is obtained by the on-board sensing device; the vehicle attribute information includes the vehicle's size, size, color, assembly relationship, brand and model, and the license plate number information is obtained by the roadside sensing device and the cloud platform; the road condition information is the road surface condition, road section speed limit, road construction status and vehicle information of the current vehicle, which is obtained by the roadside sensing device and the cloud platform; the environmental condition information is the environmental information, including weather conditions and obstacle conditions, which is obtained by the roadside sensing device and the cloud platform.

[0035] In the above technical solution, the on-board sensing equipment includes GPS, in-vehicle sensors, radar and on-board visual sensors; the roadside sensing equipment includes radar, roadside visual sensors and RSU (Road Side Unit); the map includes lane information, road components, road attributes and rule information that can be quantitatively identified, wherein the lane information includes the number of lanes, lane center lines, road separation points, lane separation points and lane relationships; road components include traffic lights, traffic signs, zebra crossings, stop lines, curbs, guardrails, gantries and bridges; road attributes include the number of lanes, lane change attributes, lane line curvature / slope, lane connection relationships, lane grouping, traffic areas, areas of interest, acceleration points and braking points; rule information includes lane speed limits, highway toll information, and traffic restriction and license plate restriction information.

[0036] In the above technical solution, the virtual model simulation data is generated by the construction module of the virtual model in the digital transportation twin layer. The construction module of the virtual model in the digital transportation twin layer specifically includes a geometric model establishment module, a physical property simulation module, a behavior model establishment module and a rule model establishment module:

[0037] The geometric model building module uses three-dimensional modeling software to build the geometric model of the physical entities involved in the digital twin model by importing components and establishing the node relationship of the model, including roads, roadside facilities and vehicles, and imports the geometric parameters (such as contour shape, size, position) and assembly relationship (such as the wheel hierarchical relationship of the vehicle model) of the physical entity from the vehicle attribute information obtained by the cloud platform and roadside facilities, so that it has temporal and spatial consistency with the physical entity equipment. At the same time, the rendering of the detail level makes the geometric model visually closer to the physical entity; it realizes high-precision, high-fidelity, and high-visualization geometric modeling of the physical traffic entity, provides a basic morphological expression for the digital twin model, and provides the necessary data support for subsequent physical property simulation, behavior simulation and rule constraints, and realizes the temporal and spatial consistency of the physical traffic entity, that is, the physical entity in the virtual model is consistent with the physical entity in the real world in terms of position, direction, size, shape, etc., thereby improving the credibility and effectiveness of the digital twin model; it realizes the rendering of the detail level of the physical traffic entity, so that the physical entity in the virtual model is visually closer to the physical entity in the real world, thereby improving the realism and aesthetics of the digital twin model;

[0038] The physical property simulation module utilizes a 3D physics simulation engine to add physical properties to the physical entities in the digital twin model, including the weight, inertia, friction coefficient, elasticity, and stiffness of each vehicle component. To account for the interactions between different vehicle components, the module simulates and characterizes vehicle speed, acceleration, steering angle, and road slope, curvature, and friction from both macro and micro perspectives. To more accurately simulate real-world conditions by accounting for collisions and interactions with other objects, the module integrates collision body properties into the system, representing the physical quantities of specific physical entities through graphical and numerical representations. The physical space represents the entities and scenes within the real-world transportation system environment, such as roads, vehicles, traffic lights, and buildings. In the digital transportation twin system, sensors collect information about these entities and map it to the virtual model for simulation and analysis. This is the physical transportation entity layer.

[0039] The above technical solution achieves a comprehensive digital representation of physical traffic entities, comprehensively modeling multiple aspects of the physical entity. After mapping and simulation, the digital traffic twin layer can obtain a virtual model that highly reproduces the physical entity. It is used to simulate and optimize the operating status of the traffic system, thereby providing support for areas such as traffic management and assisted driving. It realizes dynamic feedback and collaborative optimization of physical traffic entities, synchronizing real-world physical traffic perception information into the virtual model to update the virtual model's status in real time. Information from the physical traffic entity layer is mapped into the digital traffic twin layer's visualized 3D model, physical property simulation model, behavior model, and rule model. The real traffic environment is simulated in the model to facilitate various scenario simulations and analyses. It realizes intelligent guidance and services for physical traffic entities, and adjusts assisted driving services accordingly based on the optimization of the virtual model, improving their efficiency and safety.

[0040] The behavioral model building module maps the vehicle's behavior in different situations, including going straight, turning left, turning right, overtaking, and changing lanes. It analyzes the dynamic behavior of the vehicle and driver, including acceleration, braking, steering, and overtaking, and corrects and optimizes the dynamic behavior of the vehicle and driver based on real-time data.

[0041] The rule model building module is used to establish the constraint rules for vehicle driving in the digital twin model based on historical correlation data, experience, and knowledge base, and add traffic rules, safety rules, and driver behavior rules to ensure the safety and effectiveness of vehicle driving.

[0042] In the above technical solution, the interactive layer is connected to establish a two-way connection between physical traffic entities, digital traffic twins, and assisted driving guidance services, enabling two-way feedback and collaborative optimization; the virtual model is dynamically adjusted according to the changes in the physical entity to ensure its accuracy and timeliness, and to achieve data synchronization between the physical entity and the virtual model; the assisted driving service is adjusted accordingly based on the prediction and optimization of the virtual model to improve its efficiency and safety, and to achieve data transmission between the virtual model and the assisted driving service; the physical entity is guided according to the instructions and suggestions of the assisted driving service to improve its behavior and status, and to achieve data feedback between the assisted driving service and the physical entity.

[0043] The traffic data center layer, serving as a vehicle for interaction between modules within the digital twin model, preprocesses multi-source heterogeneous data, including parsing, cleaning, fusion, and packaging. It also stores and manages the system's driving environment perception data, virtual model simulation data, and driving guidance service data. Driving environment perception data includes vehicle posture, status, and attribute information, as well as road and environmental information. Virtual model simulation data includes various numerical information generated during the construction and operation of the twin space across four dimensions: 3D visualization models, physical property simulation models, behavioral models, and rule models. Driving guidance service data includes various types of reminders or instructions for weather services, event guidance services, lane keeping services, blind spot monitoring services, forward collision warning services, and speed limit guidance services. This layer parses, cleans, fuses, and packages multi-source heterogeneous data, improving data quality and availability and providing effective input for the construction and operation of the digital twin model. It also ensures the storage and management of driving environment perception data, virtual model simulation data, and driving guidance service data, ensuring data security and integrity and providing reliable support for updating and optimizing the digital twin model. A traffic information platform that integrates multiple types of data has been implemented, which can provide the required data for each module in the digital twin model and forward it through the connection interaction layer, realizing information sharing between the physical traffic entity layer, the digital traffic twin layer and the assisted driving guidance service layer.

[0044] In the above technical solution, the driving guidance service operation data refers to different types of reminders or instructions provided by the assisted driving guidance service layer. Generated by the assisted driving guidance service layer, this data includes the real-time status and operational information of various driving guidance services. This data can be fed back to the traffic data center layer for processing and operates synchronously with the physical traffic entity layer and the digital traffic twin layer. The purpose of this data is to provide real-time feedback and decision-making support to the traffic data center layer and the assisted driving guidance service layer.

[0045] In the above technical solution, the connection interaction layer is used to forward the digital twin data of the traffic data center layer by forwarding the digital twin data to the digital traffic twin layer, the physical traffic entity layer, and the assisted driving guidance service layer, thereby achieving two-way communication, data synchronization between the physical entity and the virtual model, data transmission between the virtual model and the assisted driving service, and data feedback between the assisted driving service and the physical entity. It achieves data synchronization between the physical entity and the virtual model, and can reflect the behavior and status of the physical entity in real time in the digital world, improving the accuracy and timeliness of the virtual model. It achieves data feedback between the assisted driving service and the physical entity, and can guide the physical entity according to the instructions and suggestions of the assisted driving service to improve its behavior and status. It realizes a cyber-physical system that integrates physical traffic entities, digital traffic twins, and assisted driving services, and can provide effective, safe, and convenient intelligent driving services in complex, dynamic, and changing traffic environments.

[0046] In the above technical solution, the assisted driving guidance services include providing drivers with vehicle weather guidance, lane keeping services, blind spot monitoring services, forward collision warning services, incident guidance services, speed guidance services, and beyond-visual-range guidance services. These services provide drivers with real-time, accurate, and safe driving information and advice, helping them navigate complex, dynamic, and ever-changing traffic environments and improving driving safety and efficiency. The system utilizes digital twin technology to achieve high-fidelity mapping and simulation of physical traffic entities, as well as comprehensive perception and analysis of the driving environment.

[0047] In the above technical solution, the method for constructing a visualized 3D model in the virtual model is as follows: using 3D modeling software, by importing components and establishing node relationships of the model, a geometric model of the physical entities involved in the digital twin model is established, including roads, roadside facilities, and vehicles. The geometric parameters (such as outline shape, size, position) and assembly relationships (such as the wheel hierarchy of the vehicle model) of the physical entities are imported from the vehicle attribute information obtained by the cloud platform and roadside facilities, so that they have temporal and spatial consistency with the physical entity equipment. At the same time, the rendering of the level of detail can make the geometric model visually closer to the physical entity.

[0048] The method for constructing a physical property simulation model in the virtual model is as follows: using a three-dimensional physical simulation engine, physical properties are added to the physical entities in the digital twin model, including the weight, inertia, friction coefficient, elasticity, and stiffness of each vehicle component. To account for the interactions between different vehicle components, the vehicle's speed, acceleration, steering angle, as well as the road's slope, curvature, and friction are simulated from both macroscopic and microscopic properties. To account for collisions and interactions between the vehicle and other objects and more accurately simulate real-world conditions, the collision body properties are integrated into the digital twin-driven intelligent assisted driving guidance system, and the physical quantities of specific physical entities are expressed in graphical and numerical form.

[0049] The behavioral model in the virtual model is constructed by mapping the state of the vehicle's behavior (including going straight, turning left, turning right, overtaking, and changing lanes), analyzing the dynamic behavior of the vehicle and driver (including acceleration, braking, steering, and overtaking), and making corrections and optimizations based on real-time data. Real-time data refers to the dynamic behavior data of the vehicle and driver, including information such as vehicle speed, acceleration, steering angle, and position, as well as driver behavior data such as braking, acceleration, and steering.

[0050] The rule model in the virtual model is used to establish the constraint rules for vehicle driving in the digital twin model; based on historical correlation data, experience data, and knowledge base, traffic rules, safety rules, and driver behavior rules are added to ensure the safety and effectiveness of vehicle driving.

[0051] A digital twin-driven intelligent assisted driving guidance method includes the following steps:

[0052] Step 1: Download, read, and parse map data. Map data is downloaded from the data acquisition and perception devices at the physical traffic entity layer to the traffic data center. This map data describes the topology, geometry, and logical properties of the road network. The data center processes the map data and parses it using a parser. The parser provides interfaces and methods to access various elements of the map data (lane information, road components, road attributes, and regulation information) and converts them into a data structure that can be processed by a computer.

[0053] Step 2: Discretize the road network data. For each parsed road data, in the traffic data center layer, different degrees of discretization are performed according to the type of road centerline. Road centerlines are divided into three types: straight segments, curved segments, and circular segments. Different types of road centerlines are discretized using different methods. For straight segments, the distance between the starting point and the end point is directly discretized uniformly. For curved segments, interpolation discretization is performed based on the length and curvature of the curve. For circular segments, the arc is discretized into multiple equally divided points. These points are then stored using a spatial index structure called a KD tree, which divides the space according to their latitude and longitude coordinates to speed up the search for the nearest point when matching the vehicle's road.

[0054] Step 3: Match the road where the vehicle is located. The traffic data center layer forwards these discretized data to the digital traffic twin layer through the connection interaction layer. In the digital traffic twin layer, the vehicle's posture information is used to simulate the vehicle's posture in the virtual model. The vehicle posture simulation includes: based on the vehicle's posture information, setting the vehicle's local coordinate system position in the digital traffic twin layer to achieve vehicle position transformation; based on the vehicle's pitch angle, roll angle and heading, setting the Euler angle of the vehicle's local coordinate to achieve vehicle posture transformation; then, based on the vehicle's position in the digital traffic twin layer, using the neighboring point matching algorithm, calculate the closest point of the vehicle to the road centerline in real time to find the road that best matches it; this process uses the spatial index structure KD tree to retrieve road data in the map, and uses the discretized points on the centerline of each road in the map as data points of the KD tree. The nearest neighbor search algorithm of the KD tree is used to quickly find the nearest road data point and match it to the road to which it belongs;

[0055] Step 4: Lane departure warning. For the matched nearest point and the road it belongs to, the difference between it and the vehicle coordinates and direction angle is calculated in the digital traffic twin layer. These differences include lateral distance, longitudinal distance, and direction angle error. The lateral distance refers to the projection distance from the nearest point to the vehicle coordinates in the direction perpendicular to the road centerline. The longitudinal distance refers to the projection distance from the nearest point to the vehicle coordinates in the direction parallel to the road centerline. The direction angle error refers to the angle between the road centerline direction angle at the nearest point and the vehicle direction angle. Based on these differences, it is determined whether lane departure has occurred. If lane deviation has occurred, the driver is provided with a warning in the assisted driving guidance service layer according to the degree and type of lane deviation. Corresponding warnings are issued: if the lateral distance exceeds a threshold, or if the angular error exceeds a threshold, a lane departure is considered to have occurred. Lane departures are categorized as active and passive. Active lane departure occurs when the driver intentionally changes the vehicle's lane. For active lane departures, the system does not issue a warning. Passive lane departure occurs when the driver unconsciously deviates from the original lane. For passive lane departures, the system issues different levels of visual and audible warnings based on the magnitude and rate of change of the lateral distance and angular error. If the lateral distance or angular error is small and the rate of change is low, a low-level warning is issued; if the lateral distance or angular error is large and the rate of change is high, a high-level warning is issued. The present invention utilizes high-precision maps and on-board sensors to locate the vehicle's position on the road in real time. When the vehicle deviates from its current lane, the driver is alerted to correct the direction, ensuring the vehicle remains in the correct lane. This design can promptly detect whether the vehicle has deviated from its original lane, assess the extent and type of lane deviation, and alert the driver to take corrective measures to prevent or mitigate accidents caused by lane departure. It uses high-precision maps and on-board sensors to locate the vehicle's position on the road in real time, calculates the difference between the vehicle and the road centerline, determines lane departure, and gives different levels of warning signals according to different situations, allowing the driver to adjust the vehicle direction in time and stay in the correct lane.

[0056] In the aforementioned technical solution, the forward collision warning service within the assisted driving guidance service layer is designed to provide the driver with timely warning signals by monitoring objects in front of the vehicle in real time, thereby preventing or mitigating the impact of head-on collisions. Using digital twin data, the digital traffic twin layer uses a ray detection method to emit multiple rays in all directions from the vehicle itself as the origin to detect interactions with virtual models in the scene. The angle, range, and frequency of the rays can be adjusted to suit different scenarios based on the vehicle type, traffic environment, and actual needs. By comparing the ray detection results with preset obstacle types, various obstacles within the safe range in front of the vehicle are identified. A Gaussian plane coordinate system is established in the digital traffic twin layer to convert longitude and latitude coordinates into plane coordinates. The position and speed of the vehicle itself and other vehicles in the plane coordinate system are obtained. The relative heading angle between the two vehicles is used to determine whether there is a collision risk. If so, the time to collision is calculated using the TTC collision distance measurement algorithm and compared with a preset threshold. If the time to collision is less than the threshold, a warning of varying severity is issued. This design can promptly detect obstacles in front of the vehicle, assess the collision risk, and prompt the driver to take evasive measures, thereby preventing or mitigating the impact of a head-on collision. The design uses digital twin technology to achieve high-precision perception and simulation of objects in front of the vehicle, as well as accurate calculation of collision time, thereby providing the driver with real-time, effective and safe warning signals.

[0057] A Gaussian plane coordinate system is established in the digital traffic twin layer, and the longitude and latitude coordinates in the driving environment perception data forwarded through the connection interaction layer are converted into plane coordinates through coordinate conversion technology; the longitude and latitude information of the own vehicle and other vehicles are converted into digital space coordinates (x1, y1), (x2, y2), and the speed is converted into The vehicle heading angles α1 and α2 of the two vehicles are the clockwise angles between the true north direction and the direction of travel of the vehicle, which can be directly obtained by the on-board equipment, where α1, α2∈[0,360°), β1 is the clockwise angle between the true north direction with the own vehicle as the origin and the line connecting the centers of the two vehicles, and β2 is the clockwise angle between the true north direction with the other vehicle as the origin and the line connecting the centers of the two vehicles, which are obtained by the vehicle coordinates in the plane coordinate system, where β1, β2∈[0,360°), θ i =α i -β i (i=1,2); Based on the relationship between θ1 and θ2, the collision types are divided into:

[0058] Head-on collision: If two vehicles are parallel and traveling towards each other, and |θ1-θ2|≤δ, there is a risk of a head-on collision.

[0059] Rear-end collision: When two vehicles are parallel and traveling in the same direction, and |θ1-θ2|∈[180°-δ,180+δ] is satisfied, there is a risk of rear-end collision. Since it is difficult for vehicles to be completely parallel in actual driving, δ is set to 5° to meet the requirements of actual driving scenarios.

[0060] If there is a risk of frontal collision or rear-end collision, the TTC ranging collision algorithm is used to calculate the collision time T of the two vehicles in real time. tc :

[0061]

[0062] in is the velocity vector of the two vehicles in the digital space, where the relative distance L between the own vehicle and other vehicles is calculated as follows:

[0063]

[0064] Where x1 and x2 are the horizontal coordinates of the two cars in the digital space, and y1 and y2 are the vertical coordinates of the two cars in the digital space;

[0065] The collision time T tc Compare with the pre-set time threshold; According to the AEBs test standard for commercial vehicles, when T tc When it is greater than the threshold value A seconds (4.4s), it means that the vehicle is driving safely and no warning is needed; when T tc When the speed is between B and A seconds (1.4 to 4.4 seconds), it means that the vehicle needs to pay attention to the situation ahead. The system will remind the driver to slow down or keep distance through sound and images in the assisted driving guidance service. When T tc When the time is less than the threshold value B seconds (1.4s), it means that the vehicle is at high risk of collision. The system will warn the driver through sound and image in the assisted driving guidance service to emergency brake or avoid. The intensity and frequency of the alarm are related to T tc The value is inversely proportional, that is, T tc The smaller the value, the stronger and more frequent the alarm. This design improves the driver's safety awareness and reaction ability, reducing the risk and damage of head-on collisions. It enables the driver to take different measures based on different collision time and thresholds, such as slowing down, maintaining distance, emergency braking, or evasive maneuvering. This allows the driver to judge the severity and urgency of the collision based on the intensity and frequency of the alarm, improving the effectiveness and timeliness of the alarm.

[0066] This design, based on digital twin technology, leverages high-precision maps and onboard sensors to locate the vehicle's position on the road in real time, enabling real-time monitoring and identification of objects in front of the vehicle. A TTC collision algorithm calculates the time to collision between the two vehicles in real time and compares it with preset thresholds, enabling dynamic assessment and early warning of collision risk. Different time thresholds and alarm levels are set based on the AEBs testing standards for commercial vehicles, providing graded warnings and guidance to the driver.

[0067] The vehicle blind spot monitoring service of the present invention provides the driver with comprehensive field of view information by switching between different perspectives to improve driving safety; the service module includes four perspectives, namely first-person perspective, third-person perspective, bird's-eye perspective and eagle-eye (map) perspective. In the auxiliary driving guidance service layer, the digital mirror constructed in the digital traffic twin layer is displayed to the driver in real time through the service to perform all-round multi-perspective monitoring; among them, the first-person perspective supplements the field of view by eliminating blind spots, fixes the viewpoint of the virtual camera at the main driving seat, and the content displayed is the driver's personal observation content, which supplements the various blind spots in the driver's perspective, including AB column blind spots, front blind spots, inner wheel difference blind spots, such as Figure 3 As shown in the figure, the third-person perspective uses a virtual camera to observe the driving environment around the vehicle at a downward angle to enhance the observation ability within the field of view. The viewpoint follows the rear of the current vehicle and observes the driving environment around the vehicle at a downward angle, as shown in the figure. Figure 4 As shown; the top-down perspective fixes the virtual camera's viewpoint directly above the vehicle, allowing you to intuitively view close-up dangers. When nearby vehicles are within the safety warning range, it automatically switches to this perspective to display the location of the danger, such as Figure 5 As shown; Eagle Eye (map) perspective is used to expand the field of view and display the road condition information beyond the visual range in the form of a small map, including road shape and traffic conditions, such as Figure 6 As shown; through multi-perspective control technology, the driver can switch between different perspectives to observe the surrounding environment, realizing real-time observation of 360-degree panoramic view and blind spot vision, enabling the driver to obtain a comprehensive driving field of view and improve the driver's driving safety factor.

[0068] The weather guidance service of the present invention is intended to provide drivers with driving rules guidance under different weather conditions to help drivers better cope with different weather conditions; a weather system is constructed in the digital traffic twin layer, and the current weather information and vehicle posture information are obtained in real time based on the environmental condition information forwarded by the connection interaction layer, and the weather conditions at the current vehicle location are obtained. Based on the weather conditions obtained in real time, by controlling the changes in the weather system of the digital traffic twin layer, different weather effects are rendered in the digital traffic twin layer according to different weather conditions, and rain removal, fog removal, and snow removal functions are provided to eliminate the field of vision obstruction caused by bad weather and provide the driver with driving rules guidance. The current weather conditions are displayed to the driver in the auxiliary driving guidance service layer, and the driver is provided with driving rules guidance on how to avoid traffic accidents caused by skidding and extended braking distance; in addition, the weather guidance service also gives the driver a speed limit reminder in accordance with the provisions on speed limits in bad weather in the "Road Traffic Safety Law of the People's Republic of China".

[0069] The speed limit guidance service of the present invention matches the current lane in the digital traffic twin layer based on the speed limit information of the road parsed from the high-precision map forwarded in the connection interaction layer, obtains the current speed limit information, and displays the speed limit value of the current road in the assisted driving guidance service layer; if the road does not have corresponding speed limit information in the high-precision map, the speed limit guidance service will display the speed limit value in accordance with the speed limit table commonly used in cities and suburbs and based on the road speed limit regulations; the service provides speed limit reminders based on the actual vehicle speed to help drivers comply with speed limit regulations.

[0070] The event guidance service of the present invention provides the driver with actual video information perceived by the on-board camera and roadside perception equipment from the physical traffic entity layer, which is forwarded by the connection interaction layer in the assisted driving guidance service; if a traffic accident or traffic jam occurs ahead, the driver can switch to the real camera perspective of the on-board camera and roadside perception equipment for observation, thereby improving the driver's control over the road. The real and virtual cameras interact with each other to provide the driver with richer driving information.

[0071] To avoid traffic accidents caused by information loss, such as visual obstruction and inability to identify the direction of sound sources due to vehicle sound insulation, this invention considers using digital twins as a driver and multi-source information fusion as a guide to provide drivers with rich decision-making information from different levels, thereby achieving real-time and safe driving guidance services.

[0072] The present invention enables the driver to gain insight into the real world with the help of the twin world, increases the driver's understanding and response ability to the driving environment, realizes human-machine co-driving, and provides safe service guidance for the driver.

[0073] Digital twin technology can adapt to different scenarios and needs. It can not only provide safety tips and avoidance strategies when visibility is low, but also provide weather services, event guidance services, lane keeping services, vehicle blind spot monitoring services, forward collision warning services and speed limit guidance services in other situations.

[0074] The contents not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A digital twin-driven intelligent assisted driving guidance system, characterized by: It includes the physical traffic entity layer, the digital traffic twin layer, the connection interaction layer, the traffic data center layer, and the assisted driving guidance service layer; The physical traffic entity layer includes objective physical entities involved in the real traffic environment and related data acquisition and perception equipment, which are used to collect and perceive perception data of the driving environment; The traffic data center layer is used to receive, store and process digital twin data to drive the synchronous operation of the data-driven physical traffic entity layer, the digital traffic twin layer and the assisted driving guidance service layer. The digital twin data includes driving environment perception data, virtual model simulation data and driving guidance service operation data; The connection interaction layer is used to forward the digital twin data of the traffic data center layer, and establish a two-way connection between the physical traffic entity layer, the digital traffic twin layer and the assisted driving guidance service layer through data synchronization and transmission; The digital traffic twin layer is a digital mirror of the physical entities in the physical traffic entity layer. The digital traffic twin layer maps the physical entities in the physical traffic entity layer to obtain a virtual model. The virtual model includes a visual three-dimensional model, a physical property simulation model, a behavior model, and a rule model. Driven by the digital twin data in the traffic data center layer forwarded by the connection interaction layer, the virtual model reflects the behavior and status of the physical entities in the physical traffic entity layer in real time, thereby realizing the simulation of the physical entities in the physical traffic entity layer and synchronizing the virtual model simulation data generated by the simulation to the traffic data center layer through the connection interaction layer; The assisted driving guidance service layer is a collection of assisted driving guidance services. The assisted driving guidance service layer uses the driving environment perception data and virtual model simulation data in the digital twin data forwarded by the connection interaction layer to provide assisted driving guidance services.

2. The digital twin-driven intelligent assisted driving guidance system according to claim 1, characterized in that: The physical entities in the physical traffic entity layer are the basic objects of the virtual model in the digital twin assisted driving system, including personnel, roads, roadside facilities, vehicles and sensors. The data acquisition and perception equipment in the physical traffic entity layer includes vehicle-mounted perception equipment, roadside perception equipment, cloud platform and map. The perception data of the driving environment are obtained through these data acquisition and perception equipment; the perception data of the driving environment includes the vehicle's posture information, status information and attribute information, road condition information and environmental condition information, wherein the vehicle's posture information is the position direction and angle information of the vehicle in three-dimensional space, and the vehicle's posture information includes longitude, latitude, altitude, pitch angle, roll angle and heading information, which is obtained by the vehicle-mounted perception equipment and roadside sensing devices; vehicle status information refers to the vehicle's operating status and performance parameter information, including engine speed, turn signal status, accelerator pedal position, brake pedal position, steering wheel angle and turn signal status, which are obtained by on-board sensing devices; vehicle attribute information includes vehicle size, size, color, assembly relationship, brand and model, and license plate number information is obtained by roadside sensing devices and the cloud platform; road condition information refers to the road surface condition, road section speed limit, road construction status and vehicle information of the current vehicle, which are obtained by roadside sensing devices and the cloud platform; environmental condition information refers to environmental information, including weather conditions and obstacle conditions, which are obtained by roadside sensing devices and the cloud platform.

3. The digital twin-driven intelligent assisted driving guidance system according to claim 2, characterized in that: The on-board sensing equipment includes GPS, in-vehicle sensors, radar, and on-board vision sensors; the roadside sensing equipment includes radar, roadside vision sensors, and RSU; the map includes lane information, road components, road attributes, and rule information that can be quantitatively identified, wherein lane information includes the number of lanes, lane centerlines, road separation points, lane separation points, and lane relationships; road components include traffic lights, traffic signs, zebra crossings, stop lines, curbs, guardrails, gantries, and bridges; road attributes include the number of lanes, lane change attributes, lane line curvature / slope, lane connection relationships, lane grouping, traffic areas, areas of interest, acceleration points, and braking points; Rule information includes lane speed limits, highway toll information, and traffic restrictions and license plate restrictions.

4. The digital twin-driven intelligent assisted driving guidance system according to claim 1, characterized in that: The virtual model simulation data is generated by the construction module of the virtual model in the digital traffic twin layer. The construction module of the virtual model in the digital traffic twin layer specifically includes a geometric model establishment module, a physical property simulation module, a behavior model establishment module and a rule model establishment module: The geometric model building module uses 3D modeling software to build the geometric model of the physical entities involved in the digital twin model by importing components and establishing the node relationship of the model. The geometric parameters and assembly relationships of the physical entities are imported from the vehicle attribute information obtained from the cloud platform and roadside facilities, so that they have temporal and spatial consistency with the physical entity equipment. At the same time, the rendering of the level of detail makes the geometric model visually closer to the physical entity. The physical property simulation module uses a 3D physical simulation engine to add physical properties to physical entities in the digital twin model; The behavior model building module maps the vehicle's behavior in different situations, analyzes the dynamic behavior of the vehicle and driver, and corrects and optimizes the dynamic behavior of the vehicle and driver based on real-time data; The rule model establishment module is used to establish the constraint rules for vehicle driving in the digital twin model, including traffic rules, safety rules, and driver behavior rules.

5. The digital twin-driven intelligent assisted driving guidance system according to claim 1, characterized in that: The driving guidance service operation data is different types of reminder or instruction information provided by the auxiliary driving guidance service layer.

6. The digital twin-driven intelligent assisted driving guidance system according to claim 1, characterized in that: The connection interaction layer is used to forward the digital twin data of the traffic data center layer: forwarding the digital twin data to the digital traffic twin layer, the physical traffic entity layer and the assisted driving guidance service layer to achieve two-way connection, realize data synchronization between the physical entity and the virtual model, data transmission between the virtual model and the assisted driving service, and data feedback between the assisted driving service and the physical entity.

7. The digital twin-driven intelligent assisted driving guidance system according to claim 1, characterized in that: The assisted driving guidance service includes providing the driver with vehicle weather guidance service, lane keeping service, vehicle blind spot monitoring service, forward collision warning service, event guidance service and speed guidance service beyond visual range guidance service.

8. The digital twin-driven intelligent assisted driving guidance system according to claim 1, characterized in that: The method for constructing a visual 3D model in the virtual model is as follows: using 3D modeling software, by importing components and establishing model node relationships, the geometric model of the physical entities involved in the digital twin model, including roads, roadside facilities, and vehicles, is established. The geometric parameters and assembly relationships of the physical entities are imported from the vehicle attribute information obtained from the cloud platform and roadside facilities, so that they have temporal and spatial consistency with the physical entity equipment. At the same time, the rendering of the level of detail can make the geometric model visually closer to the physical entity. The method for constructing a physical property simulation model in the virtual model is as follows: using a 3D physics simulation engine, physical properties are added to the physical entities in the digital twin model; to account for the interactions between different vehicle components, the vehicle's speed, acceleration, steering angle, as well as the road's slope, curvature, and friction are simulated; to account for collisions and interactions between the vehicle and other objects and more accurately simulate real-world conditions, the collision body properties are integrated into the digital twin-driven intelligent assisted driving guidance system, and the physical quantities of specific physical entities are expressed in graphical and numerical form; The behavior model in the virtual model is constructed by mapping the vehicle's behavior, analyzing the dynamic behavior of the vehicle and the driver, and making corrections and optimizations based on real-time data; The rule model in the virtual model is used to establish the constraint rules for vehicle driving in the digital twin model; based on historical correlation data, experience data, and knowledge base, traffic rules, safety rules, and driver behavior rules are added to ensure the safety and effectiveness of vehicle driving.

9. An intelligent assisted driving guidance method driven by a digital twin of the system according to claim 1, characterized in that: It includes the following steps: Step 1: Download map data from the data collection and perception devices at the physical traffic entity layer to the traffic data center. This map data is used to describe the topology, geometry, and logical attributes of the road network. The map data is processed in the traffic data center and parsed using the relevant parser. Through the interfaces and methods provided by the parser, the various elements of the map data are accessed and converted into a data structure that can be processed by the computer. Step 2: For each parsed road data, discretize it to different degrees according to the type of road centerline in the traffic data center layer; Step 3: The traffic data center layer forwards this discretized data to the digital traffic twin layer through the connection interaction layer. In the digital traffic twin layer, the vehicle's posture information is used to simulate the vehicle's position in the virtual model. Then, based on the vehicle's position in the digital traffic twin layer, a neighboring point matching algorithm is used to calculate the closest point to the road centerline in real time and find the road that best matches it. This process uses the spatial index structure KD tree to retrieve road data from the map. The discretized points on the centerline of each road in the map are used as data points in the KD tree. The KD tree's nearest neighbor search algorithm is used to quickly find the nearest road data point and match it to the road it belongs to. Step 4: For the matched nearest point and the road it belongs to, calculate the difference between it and the vehicle coordinates and direction angle in the digital traffic twin layer; Based on these differences, it is determined whether lane deviation has occurred. If lane deviation has occurred, the driver will be given corresponding warnings at the assisted driving guidance service layer based on the degree and type of lane deviation.

10. The digital twin-driven intelligent assisted driving guidance method according to claim 9, characterized in that: The forward collision warning service of the assisted driving guidance service in the assisted driving guidance service layer is designed to provide the driver with timely warning signals by monitoring objects in front of the vehicle in real time to prevent or mitigate the impact of a head-on collision; Using digital twin data, the system uses a ray detection method in the digital traffic twin layer to emit multiple rays in all directions from the vehicle itself to detect interactions with virtual models in the scene. The angle, range, and frequency of the rays are adjusted based on the vehicle type, traffic environment, and actual needs to adapt to different scenarios. By comparing the ray detection results with preset obstacle types, various obstacles within the safe range in front of the vehicle are identified. A Gaussian plane coordinate system is established in the digital traffic twin layer to convert longitude and latitude coordinates into plane coordinates. The position and speed of the vehicle and other vehicles in the plane coordinate system are obtained. The relative heading angle between the two vehicles is used to determine whether there is a collision risk. If so, the collision time is calculated using the TTC collision distance measurement algorithm and compared with a preset threshold. If the collision time is less than the threshold, a warning of varying degrees is issued. A Gaussian plane coordinate system is established in the digital traffic twin layer, and the longitude and latitude coordinates in the driving environment perception data forwarded through the connection interaction layer are converted into plane coordinates through coordinate conversion technology; the longitude and latitude information of the own vehicle and other vehicles are converted into digital space coordinates (x1, y1), (x2, y2), and the speed is converted into The vehicle heading angles α1 and α2 of the two vehicles are the clockwise angles between the true north direction and the direction of travel of the vehicle, which can be directly obtained by the on-board equipment, where α1, α2∈[0,360°), β1 is the clockwise angle between the true north direction with the own vehicle as the origin and the line connecting the centers of the two vehicles, and β2 is the clockwise angle between the true north direction with the other vehicle as the origin and the line connecting the centers of the two vehicles, which are obtained by the vehicle coordinates in the plane coordinate system, where β1, β2∈[0,360°), θ i =α i -β i , i=1,2; according to the relationship between θ1 and θ2, the collision types are divided into: Head-on collision: If two vehicles are parallel and traveling towards each other, and |θ1-θ2|≤δ, there is a risk of a head-on collision. Rear-end collision: When two vehicles are parallel and traveling in the same direction, and |θ1-θ2|∈[180°-δ,180+δ] is satisfied, there is a risk of rear-end collision; If there is a risk of frontal collision or rear-end collision, the TTC ranging collision algorithm is used to calculate the collision time T of the two vehicles in real time. tc : in is the velocity vector of the two vehicles in the digital space, where the relative distance L between the own vehicle and other vehicles is calculated as follows: Where x1 and x2 are the horizontal coordinates of the two cars in the digital space, and y1 and y2 are the vertical coordinates of the two cars in the digital space; The collision time T tc Compare with the pre-set time threshold; According to the AEBs test standard for commercial vehicles, when T tc When it is greater than the threshold value A seconds, it means that the vehicle is driving safely and no warning is needed; when T tc Between B and A seconds, it means that the vehicle needs to pay attention to the situation ahead, and the system will remind the driver to slow down or keep distance through sound and images in the assisted driving guidance service; when T tc When the time is less than the threshold value B seconds, it indicates that the vehicle is at high risk of collision. The system will warn the driver through sound and image in the assisted driving guidance service to make emergency braking or avoidance. The intensity and frequency of the alarm are related to T tc The value is inversely proportional, that is, T tc The smaller the value, the stronger and more frequent the alarm.

Images