Automatic driving system based on real-time holographic modeling and dynamic shielding compensation and vehicle

Abstract

The invention discloses an automatic driving system based on real-time holographic modeling and dynamic shielding compensation and a vehicle, and aims to solve the problem of complicated scene perception. The system comprises a real-time holographic environment modeling module for constructing a dynamic 3D environment model; the dynamic shielding compensation module outputs compensated environment data; the sensing data fusion module is used for integrating multi-source data; the high-precision positioning and mapping module is used for generating a centimeter-level environment map; the edge computing node is used for cooperatively processing fusion data; the AI self-adaptive decision making system generates a driving instruction, and the decision making efficiency is optimized and improved through a satellite auxiliary decision making unit; the V2Xcommunication module supports low-delay and high-security data interaction; the execution control unit is used for accurately controlling the vehicle; an FPGA + GPU parallel computing framework is adopted, and decision making and control are supported after modeling and compensation data are fused. According to the invention, optimal decision making and high-precision real-time control in a complex environment can be realized, and development of an automatic driving technology is promoted.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an autonomous driving system and vehicle based on real-time holographic modeling and dynamic occlusion compensation. Background Art

[0002] An autonomous driving system is a complex system that integrates functions such as environmental perception, decision-making and planning, and control execution. With the continuous development of artificial intelligence technologies such as deep learning and reinforcement learning, autonomous driving systems will have stronger environmental perception capabilities, more accurate road prediction capabilities, and more intelligent decision-making and planning capabilities. Autonomous driving technology has made significant progress in recent years. Mainstream systems such as Tesla's FSD (Full Self-Driving) and Huawei's ADS (Autonomous Driving Solution) have achieved partial autonomous driving functions from L2 to L3 levels. These systems mainly rely on rule engines (Rule-Based Engine) and static physics modeling (Static Physics-Based Modeling), combined with high-precision maps and sensor data for environmental perception and decision-making. However, with the complexity and diversification of autonomous driving application scenarios, existing technologies have gradually exposed the following limitations:

[0003] (1) Lack of real-time performance. Existing autonomous driving systems rely heavily on predefined high-precision maps and static physical rules for environmental perception and decision-making. This static modeling approach cannot effectively respond to dynamic environmental changes. For example, temporary diversions or construction areas cannot be updated to predefined maps in real time, resulting in the vehicle being unable to correctly plan its path.

[0004] (2) The accuracy of environmental modeling is limited. Existing systems mainly rely on 2D visual data and 3D lidar point clouds for environmental modeling. 2D visual data lacks depth information, while 3D lidar point clouds have sparsity problems in long-range target detection, resulting in limited modeling accuracy.

[0005] (3) Insufficient multimodal data fusion. Existing systems fail to fully utilize the data advantages of multimodal sensors (such as infrared and millimeter-wave radars), resulting in insufficient integrity and accuracy of environmental models. Static modeling cannot reflect environmental changes in real time, such as the dynamic status of moving vehicles, pedestrians, or traffic signs.

[0006] (4) The dynamic obstacle prediction capability is insufficient. When faced with sudden occlusions (such as a sudden braking of a vehicle ahead or a sudden intrusion of a pedestrian), the existing system’s prediction capability has significant flaws and its ability to process occluded areas is weak. After an occlusion occurs, the system cannot quickly generate a compensation strategy, resulting in decision delays and safety hazards.

[0007] (5) The decision-making system relies too much on empirical rules. The decision-making modules of existing autonomous driving systems are mainly based on defined rules. The rule engine cannot adapt to new scenarios that have not been seen before, such as special traffic signs or rare road layouts. The system cannot dynamically adjust the decision-making strategy according to real-time environmental changes, resulting in poor performance in complex scenarios.

[0008] (6) Computing efficiency is limited. The existing system mainly adopts the traditional computing architecture of CPU+GPU, which has high computing latency. The perception and decision computing latency in complex environments is usually higher than 10ms, which makes it difficult to meet the needs of real-time driving control. The traditional architecture consumes high energy when processing large-scale sensor data, which limits the deployment capability of the on-board computing platform. The task allocation between CPU and GPU is not optimized, resulting in low computing resource utilization.

[0009] The industry has proposed some improvements to address these issues, but significant shortcomings remain: High-precision map reliance: Some systems attempt to improve environmental modeling capabilities through higher-precision map data, but map updates are infrequent, costly, and difficult to adapt to dynamic environmental changes. Deep learning applications: Some systems incorporate deep learning algorithms for object detection and classification, but model training relies on large amounts of labeled data and struggles to adapt to real-time requirements. V2X technology initiatives: Vehicle-to-everything (V2X) technology is being used to enhance environmental perception, but existing systems fail to fully leverage the predictive and decision-making potential of V2X data. Therefore, further optimizing autonomous driving systems has become a technical issue worthy of in-depth exploration in this field. Summary of the Invention

[0010] The purpose of the present invention is to provide an autonomous driving system, method, medium, equipment and vehicle based on real-time holographic modeling and dynamic occlusion compensation. By integrating technologies such as real-time holographic environment modeling, dynamic occlusion compensation, intelligent adaptive decision-making and efficient computing framework, a new dynamic intelligent autonomous driving system is constructed to achieve optimal decision-making of vehicle behavior and real-time, efficient and precise driving control, to make up for the shortcomings of existing technologies in real-time, accuracy, predictive ability and computing efficiency, and to promote the development of autonomous driving technology to a higher level.

[0011] To achieve the above object, the present invention provides the following solutions:

[0012] In a first aspect, the present invention provides an autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation, comprising:

[0013] A real-time holographic environment modeling module, which uses holographic imaging technology to build a dynamic 3D environment model based on the vehicle's real-time multimodal sensor data;

[0014] The dynamic occlusion compensation module uses Bayesian reasoning and Kalman filtering algorithms based on a dynamic 3D environment model, combined with multimodal sensor data and vehicle-to-vehicle communication data, to predict occluded areas and the motion trajectory of occluders. It also uses V2X vehicle-to-everything (V2X) data to enhance prediction accuracy, repair visual blind spots, and generate occlusion-compensated environmental data.

[0015] a sensor data fusion module for performing spatiotemporal alignment and fusion processing on the multimodal sensor data, the dynamic 3D environment model constructed by the real-time holographic environment modeling module, and the occlusion-compensated environment data obtained by the dynamic occlusion compensation module to obtain fused environment data;

[0016] High-precision positioning and mapping module, used to generate high-precision real-time environmental maps;

[0017] Edge computing nodes, used to collaboratively process the fusion environment data and computing tasks;

[0018] An AI adaptive decision-making system, which generates autonomous driving decision instructions based on fused environmental data and high-precision real-time environmental maps, using deep reinforcement learning algorithms and Markov decision process modeling;

[0019] a satellite-assisted decision-making unit, configured to optimize the autonomous driving decision instructions in combination with satellite communication data;

[0020] V2X & StarV2X communication module, used to realize vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-cloud (V2C) data exchange through 5G low-latency communication (URLLC) including 5G C and 5G NR or satellite link (Starlink), and perform real-time environmental perception;

[0021] The execution control unit is used to convert autonomous driving decision instructions into vehicle behavior, control the electronic throttle, brake and steering systems, and realize autonomous driving of the vehicle;

[0022] Among them, the autonomous driving system adopts an FPGA+GPU parallel computing framework, integrating FPGA and GPU parallel computing units for multimodal sensor data processing, modeling and autonomous driving decision instruction calculation.

[0023] Furthermore, the autonomous driving system also includes a vehicle perception system and an external data interface module:

[0024] The vehicle perception system includes a multimodal sensor carried by the vehicle, which is used to collect information about the vehicle's surrounding environment; the multimodal sensor includes a lidar, a millimeter-wave radar, a camera, an ultrasonic sensor, and a GNSS / GPS positioning unit;

[0025] The vehicle perception system includes a multimodal sensor carried by the vehicle for collecting information about the vehicle's surrounding environment; the multimodal sensor includes a lidar, a millimeter-wave radar, a camera, an ultrasonic sensor, a GNSS / GPS positioning unit, and an inertial measurement unit (IMU);

[0026] The external data interface module is used to provide external supplementary data, which includes traffic signal data, cloud-based high-precision map data, V2X communication data, and real-time traffic information;

[0027] The vehicle's surrounding environment information and external supplementary data constitute the vehicle's real-time multimodal sensing data, and high-precision perception data is generated through spatiotemporal alignment and data fusion.

[0028] Furthermore, the multimodal sensor includes:

[0029] A laser radar unit configured to collect environmental point cloud data with a collection range greater than 250 meters, using a 64-line or equivalent high-resolution laser radar with a scanning frequency of ≥10 Hz;

[0030] a millimeter-wave radar unit configured to collect environmental distance data, wherein the environmental distance data has a detection range of not less than 200 meters;

[0031] a camera unit configured to collect environmental image data, wherein the resolution of the environmental image data is not less than 1920×1080 pixels and the frame rate is not less than 30 frames per second;

[0032] An ultrasonic sensor unit is configured to collect close-range obstacle data using high-power ultrasonic array technology, with a detection range of 0-8 meters and a detection accuracy of ±5 cm;

[0033] A GNSS / GPS positioning unit configured to collect vehicle position data using RTK technology, with a positioning accuracy of centimeters (2-10 cm);

[0034] The inertial measurement unit (IMU) is configured to collect vehicle acceleration, angular velocity, and attitude data with a sampling frequency of ≥100 Hz, an angular velocity measurement accuracy of ±0.01° / s, and an acceleration measurement accuracy of ±0.01g. It supports the high-precision positioning and mapping module's positioning continuity in scenarios where the GNSS signal is interrupted or blocked, and assists the dynamic occlusion compensation module's target tracking and environmental model updates.

[0035] Furthermore, the autonomous driving system further includes:

[0036] A path navigation trajectory calculation unit, configured to realize path navigation and trajectory generation based on the high-precision real-time environment map;

[0037] The path navigation trajectory calculation unit includes:

[0038] A path planning unit is configured to perform path planning based on the high-precision real-time environment map using a path planning algorithm to generate a global navigation path with path accuracy supporting centimeter-level navigation requirements;

[0039] a trajectory calculation unit configured to calculate a driving trajectory of the vehicle based on the global navigation path, wherein the trajectory calculation delay meets real-time driving requirements;

[0040] A trajectory optimization unit is configured to optimize the driving trajectory by combining the external supplementary data and the occlusion-compensated environmental data to improve trajectory continuity and collision safety, and the trajectory optimization delay supports dynamic scenarios.

[0041] Furthermore, the real-time holographic environment modeling module includes:

[0042] A data preprocessing submodule is used to preprocess multimodal sensor data, including filtering, denoising, and calibration; the multimodal sensor data includes LiDAR point cloud data, camera visual images, and millimeter-wave radar detection data;

[0043] 3D voxel grid generation submodule, which is used to convert the multimodal sensor data after data preprocessing into a 3D voxel grid model and annotate the spatial occupancy probability of the voxels;

[0044] An image enhancement submodule, configured to enhance the texture of low-resolution areas in the three-dimensional voxel grid model using a super-resolution algorithm that integrates a Transformer neural network and a generative adversarial network (GAN) to generate a high-definition holographic image;

[0045] A feature extraction submodule is used to extract multi-scale environmental features from the enhanced high-definition holographic image using a deep learning model. The multi-scale environmental features include lane lines, obstacle outlines, and dynamic target semantic labels. The multi-scale environmental features support dynamic environmental map generation, occlusion area recognition, and real-time driving parameter adjustment.

[0046] The point cloud reconstruction submodule is used to generate a holographic point cloud model with dynamic attribute annotations through a point cloud reconstruction algorithm based on the spatial correlation between the extracted multi-scale environmental features and the voxel grid;

[0047] The holographic imaging unit is used to construct a dynamic 3D environmental model of the dynamic environment around the vehicle based on the time series data of the holographic point cloud model output by the point cloud reconstruction submodule through holographic imaging technology. The dynamic 3D environmental model includes a three-dimensional spatial structure and dynamic environmental attributes. The dynamic environmental attributes cover road information, obstacle motion status and occlusion area identification. The holographic imaging unit cooperates with the dynamic occlusion compensation module and is configured to compensate for the perception blind spots in occlusion scenarios by analyzing the time series data and dynamic environmental attributes.

[0048] Furthermore, the dynamic occlusion compensation module includes:

[0049] an occlusion detection unit, configured to identify occlusion areas in the dynamic 3D environment model using a data segmentation algorithm and collaborate with the holographic imaging unit to improve the accuracy of occlusion area perception;

[0050] a data compensation unit, configured to fill the occluded area based on historical holographic data or adjacent sensor data using a Bayesian estimation algorithm;

[0051] The data fusion optimization unit is used to combine the dynamic 3D environment model and external supplementary data to optimize the filling result of the occluded area.

[0052] Furthermore, the sensor data fusion module includes:

[0053] a data preprocessing unit configured to filter and align the multimodal sensor data;

[0054] a fusion computing unit configured to perform data fusion and generate fusion environment data;

[0055] a data calibration unit configured to calibrate the fused environment data based on external supplementary data to improve the robustness of the fused environment data;

[0056] Among them, the data preprocessing unit adopts Gaussian filtering algorithm for filtering, the fusion calculation unit adopts Kalman filtering algorithm to perform data fusion, and the multimodal sensor data includes point cloud data, image data and distance data collected by the vehicle perception system, and dynamic 3D environment model data generated by the real-time holographic environment modeling module.

[0057] Furthermore, the high-precision positioning and mapping module is used to generate a high-precision real-time environment map, specifically including:

[0058] The high-precision positioning and mapping module uses data collected by the vehicle's onboard lidar, millimeter-wave radar, camera, ultrasonic sensor, GNSS / GPS positioning unit and inertial measurement unit (IMU) to perform spatiotemporal alignment and fusion using the SLAM algorithm to obtain the vehicle's position, posture and high-precision real-time environmental map. The vehicle's position and posture are transmitted to the execution control unit, and the high-precision real-time environmental map is transmitted to the AI ​​adaptive decision-making system.

[0059] Furthermore, the high-precision positioning and mapping module includes:

[0060] A positioning data acquisition unit configured to process positioning data from Global Navigation Satellite System (GNSS) data, Inertial Measurement Unit (IMU) data, environmental perception sensors, and Internet of Vehicles data, and generate high-precision, high-frequency positioning information through data fusion. The positioning information meets the accuracy requirements of centimeter-level navigation and the frequency requirements of real-time positioning;

[0061] a map generation unit configured to generate a dynamic environment map based on a positioning and mapping algorithm, in combination with the positioning information and environmental perception data from the positioning data acquisition unit, wherein the dynamic environment map includes road information and environmental features, wherein the road information supports path planning and trajectory generation, and the environmental features support dynamic environment adaptation, wherein the positioning accuracy of the dynamic environment map meets centimeter-level navigation requirements, and the update frequency supports dynamic driving scenarios;

[0062] a map optimization unit configured to optimize the dynamic environment map by combining external supplementary data and environmental perception data to improve the map's accuracy, stability, and environmental adaptability, wherein the external supplementary data includes cloud-based map data, real-time traffic information, and historical map data. The optimization includes updating road information, correcting environmental features, and calibrating occluded areas to support real-time environmental perception and autonomous driving decision-making;

[0063] The high-precision positioning and mapping module supports dynamic occlusion compensation and is configured to identify and process occluded areas based on environmental perception data, enhancing the robustness of environmental perception and the reliability of autonomous driving decisions through occlusion detection algorithms; the dynamic environmental map is further configured to support real-time environmental adaptation, including dynamically adjusting driving parameters related to road information to respond to real-time speed limit requirements and dynamic driving scenarios.

[0064] Furthermore, the edge computing node includes:

[0065] a data processing unit configured to locally process the perception data of the real-time holographic environment modeling module and the sensor data fusion module to support real-time environment perception and calculate delays to meet dynamic driving requirements;

[0066] A task allocation unit, configured to schedule perception and mapping tasks, high-reliability communication tasks, and high-computation tasks, with scheduling delays supporting real-time collaboration;

[0067] Data cache unit, configured to cache-aware and fuse data to reduce data processing latency, with cache capacity supporting efficient data access;

[0068] Among them, the edge computing nodes provide high-performance computing capabilities, support dynamic occlusion compensation and decision-making tasks, and the data transmission delay meets real-time interaction requirements.

[0069] Furthermore, the AI ​​adaptive decision-making system includes:

[0070] An environmental analysis unit, configured to analyze the dynamic 3D environmental model and the high-precision real-time environmental map based on a deep learning model, perform multi-dimensional scene classification, and identify different driving scenarios, including urban roads, highways, complex intersections, and parking lots; the deep learning model includes a convolutional neural network (CNN) and a recurrent neural network (RNN);

[0071] The Markov decision modeling module is used to build a Markov decision process model and define the state space, action space, state transition probability, and reward function. The state space includes the driving scenario, vehicle state, and environmental dynamics; the action space includes acceleration, deceleration, braking, steering, and holding strategies; the state transition probability represents the possibility of vehicle state transition based on the driving scenario and action; and the reward function evaluates driving behavior based on safety, driving efficiency, and path optimization to generate a reward value that supports real-time decision-making.

[0072] A reinforcement learning module for optimizing the Markov decision process model based on deep reinforcement learning to train the intelligent agent to generate an autonomous driving policy. The module employs a value iteration algorithm to optimize the autonomous driving policy to adapt to the dynamic 3D environment model, combines a deep neural network to extract environmental features, including obstacle locations and lane markings, and utilizes self-supervised learning to enhance policy generalization, thereby generating a policy that supports safe and efficient driving.

[0073] The vehicle behavior planning module is used to generate driving trajectories, accelerations, and obstacle avoidance paths based on the autonomous driving strategy and a dynamic 3D environment model to achieve safe and efficient vehicle control. This module uses a trajectory smoothing algorithm to generate smooth driving trajectories, an optimization algorithm to calculate accelerations to ensure driving efficiency and comfort, and a path planning algorithm to generate local obstacle avoidance paths to address dynamically blocked areas.

[0074] The AI ​​execution module is used to determine autonomous driving decision instructions based on smooth driving trajectories, optimized acceleration, and local obstacle avoidance paths.

[0075] Furthermore, the satellite-assisted decision-making unit includes:

[0076] A satellite data receiving unit configured to receive high-precision positioning data or wide-area environmental data, with data reception latency supporting real-time decision-making requirements;

[0077] a decision optimization unit configured to optimize the autonomous driving decision instructions based on the data received by the satellite data receiving unit to improve path planning accuracy or driving safety, and optimize the calculation delay of the autonomous driving decision instructions to meet dynamic driving requirements;

[0078] a data synchronization unit configured to synchronize the data received by the satellite data receiving unit with the fused environment data to enhance the real-time and consistency of the autonomous driving decision instructions;

[0079] The satellite data receiving unit adopts high-bandwidth satellite communication technology; the satellite-assisted decision-making unit supports dynamic occlusion compensation and real-time path optimization.

[0080] Furthermore, the V2X&StarV2X communication module includes:

[0081] The V2X communication unit is used to exchange data with surrounding vehicles or infrastructure through 5G C or 5G NR-V2X sidelink communication, supporting ultra-low latency data transmission;

[0082] The StarV2X communication unit provides data interaction via satellite communications, serving as a backup in the event of ground communication interruptions and supporting low-latency and high-security transmission;

[0083] a communication switching unit, configured to dynamically switch between the V2X communication unit and the StarV2X communication unit according to a communication environment to ensure communication continuity;

[0084] Among them, the communication delay between the V2X communication unit and surrounding vehicles is less than 2 milliseconds, and the transmission error with the infrastructure is less than 1 millisecond; the communication delay of the StarV2X communication unit is less than 5 milliseconds, and high-level data encryption is adopted; the switching time of the communication switching unit is less than the overall system delay requirement.

[0085] Furthermore, the FPGA+GPU parallel computing framework includes:

[0086] The task allocation module is used to dynamically allocate tasks to FPGA or GPU for execution based on the real-time requirements and computing power requirements of the computing tasks, where the total computing delay meets T total =max(T FPGA ,T GPU ), and the overall system delay is constrained to <5ms;

[0087] The data flow optimization module is used to optimize the low-latency transmission of input and output data between the FPGA and GPU using zero-copy memory mapping and pipeline batch processing technology;

[0088] FPGA, deploying customized acceleration logic implemented in hardware description language to perform multimodal sensor data preprocessing, control instruction verification, and time-sensitive algorithms;

[0089] GPU, equipped with CUDA computing cores, is used for parallel processing of deep learning model inference, high-dimensional path planning, and multi-target trajectory prediction tasks;

[0090] The real-time optimization module is used to perform real-time load balancing of FPGA and GPU computing resources through dynamic voltage and frequency adjustment and task priority preemption mechanism.

[0091] Furthermore, the execution control unit includes:

[0092] an instruction conversion unit, configured to convert the autonomous driving decision instruction into a motion control signal for the vehicle;

[0093] A behavior feedback unit, configured to monitor the vehicle's execution status and adjust the autonomous driving decision instructions;

[0094] A safety verification unit, configured to perform safety verification on the motion control signal;

[0095] The motion control signal includes a throttle control signal, a brake control signal and a steering wheel control signal.

[0096] In a second aspect, the present invention further provides an autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation, which is applied to the aforementioned autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation, and includes the following steps:

[0097] Based on the vehicle's real-time multimodal sensor data, a dynamic 3D environment model is constructed using holographic imaging technology;

[0098] Based on a dynamic 3D environmental model, the system uses Bayesian reasoning and Kalman filtering algorithms, combined with multimodal sensor data and vehicle-to-vehicle communication data, to predict occluded areas and the motion trajectory of obstructing objects. It also uses V2X vehicle-to-everything (V2X) data to enhance prediction accuracy, repair visual blind spots, and obtain occlusion-compensated environmental data.

[0099] Perform spatiotemporal alignment and fusion processing on multimodal sensor data, dynamic 3D environment model and occlusion-compensated environment data to obtain fused environment data;

[0100] Based on the fusion of environmental data and high-precision environmental maps, deep reinforcement learning algorithms and Markov decision process modeling are used to generate autonomous driving decision instructions;

[0101] Optimize autonomous driving decision instructions by combining satellite communication data;

[0102] Transmitting autonomous driving decision instructions to the execution control unit through the V2X & StarV2X communication module;

[0103] The execution control unit converts the autonomous driving decision instructions into vehicle behavior, controls the electronic throttle, brake and steering systems, and realizes the vehicle's autonomous driving;

[0104] Among them, the autonomous driving method adopts the FPGA+GPU parallel computing framework to process multimodal sensor data, modeling and autonomous driving decision instruction calculation, ensuring that the overall system delay is less than 5ms.

[0105] In a third aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by at least one processor of the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation as described above, the at least one processor implements the autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation as described above.

[0106] In a fourth aspect, the present invention also provides an electronic device comprising: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the above-mentioned autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation.

[0107] In a fifth aspect, the present invention also provides an autonomous driving vehicle equipped with the above-mentioned autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation.

[0108] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation proposed by the present invention is a new generation of dynamic intelligent autonomous driving system that integrates real-time holographic environment modeling, dynamic occlusion compensation, intelligent adaptive decision-making and efficient computing architecture optimization; the system includes a real-time holographic environment modeling module, which uses dynamic 3D modeling technology to build a dynamic 3D environment model; a dynamic occlusion compensation module, combined with Bayesian reasoning +

[0109] Kalman filtering predicts the motion trajectory of hidden targets under visual occlusion, outputs compensated environmental data, and enhances perception stability; sensor data fusion module integrates multi-source data; high-precision positioning and mapping module generates centimeter-level high-precision real-time environmental maps; edge computing nodes collaboratively process fused data; AI adaptive decision-making system, based on a dynamic 3D environmental model, adopts reinforcement learning + Markov decision process modeling method to achieve optimal decision-making for vehicle behavior, and improves decision-making efficiency through satellite-assisted decision-making unit optimization; V2X & StarV2X communication module integrates V2X & Starlink satellite communication technology to achieve vehicle network collaboration and provide low-latency, high-security V2X backup solutions, improving data fusion and prediction accuracy; execution control unit achieves precise vehicle control; FPGA + GPU parallel computing framework is used to improve computing efficiency, ensure latency is less than 5ms, and achieve true real-time driving control; modeling and compensation data are fused to support decision-making and control. This invention can achieve optimal decision-making and high-precision real-time control in complex environments, promoting the development of autonomous driving technology.

[0110] Compared to existing physical static modeling, this invention utilizes real-time holographic environment modeling, combined with mathematical optimization methods such as Bayesian reasoning, reinforcement learning, optical flow prediction, and Markov decision processes, to achieve efficient, safe, and intelligent autonomous driving decision-making, thereby improving the safety and accuracy of autonomous driving. The autonomous driving system provided by this invention offers the advantages of high precision, high stability, low latency, and high security, and is suitable for urban roads, highways, and intelligent transportation scenarios. With its superior performance and adaptability to multiple scenarios, this invention demonstrates strong application prospects and commercial value, and is expected to become a leading technology in the field of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0112] Figure 1 Schematic diagram of the architecture of the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation of the present invention;

[0113] Figure 2 This is a flowchart of the multi-sensor fusion perception system according to an embodiment of the present invention;

[0114] Figure 3 This is a workflow diagram of the real-time holographic environment modeling module according to an embodiment of the present invention;

[0115] Figure 4This is a workflow diagram of a dynamic occlusion compensation module according to an embodiment of the present invention;

[0116] Figure 5 This is a workflow diagram of the AI ​​adaptive decision-making system according to an embodiment of the present invention;

[0117] Figure 6 This is a workflow diagram of the V2X&StarV2X communication module according to an embodiment of the present invention;

[0118] Figure 7 This is a workflow diagram of the FPGA+GPU parallel computing framework according to an embodiment of the present invention;

[0119] Figure 8 This is a workflow diagram of the high-precision positioning and mapping module in an embodiment of the present invention. DETAILED DESCRIPTION

[0120] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0121] The purpose of this invention is to provide an autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation, integrating a number of cutting-edge technologies such as holographic environment modeling, dynamic occlusion compensation, intelligent adaptive decision-making and efficient computing architecture optimization.

[0122] Example 1

[0123] like Figures 1-8 As shown, the automatic driving system based on real-time holographic modeling and dynamic occlusion compensation provided by the present invention includes:

[0124] Vehicle perception system, including multimodal sensors onboard the vehicle (e.g., lidar, millimeter-wave radar, camera, infrared sensor, GNSS / GPS positioning unit, inertial measurement unit (IMU), etc.), used to collect information about the vehicle's surrounding environment;

[0125] An external data interface module is used to provide external supplementary data, including but not limited to traffic signal data (such as traffic light countdowns), cloud-based high-precision map data, V2X communication data (such as road construction information and surrounding vehicle trajectories), and real-time traffic information. The vehicle's surrounding environment information and external supplementary data constitute the vehicle's real-time multimodal sensor data. High-precision perception data (positioning accuracy of 2-10 cm, update frequency ≥10 Hz) is generated through spatiotemporal alignment and data fusion.

[0126] A real-time holographic environment modeling module, which uses holographic imaging technology to build a dynamic 3D environment model based on the vehicle's real-time multimodal sensor data;

[0127] The dynamic occlusion compensation module uses Bayesian reasoning and Kalman filtering algorithms based on a dynamic 3D environment model, combining multimodal sensor data and vehicle-to-everything (V2X) communication data (such as traffic light status and vehicle trajectory). This module predicts occluded areas and the motion trajectory of obstructing objects. It also uses V2X data to enhance prediction accuracy, repair visual blind spots, and generate occlusion-compensated environmental data. The dynamic occlusion compensation module helps enhance perception stability.

[0128] a sensor data fusion module for performing spatiotemporal alignment and fusion processing on the multimodal sensor data, the dynamic 3D environment model constructed by the real-time holographic environment modeling module, and the occlusion-compensated environment data obtained by the dynamic occlusion compensation module to obtain fused environment data;

[0129] The high-precision positioning and mapping module is used to generate high-precision real-time environmental maps. Specifically, based on the data collected by the vehicle's lidar, millimeter-wave radar, camera, ultrasonic sensor, GNSS / GPS positioning unit, and inertial measurement unit (IMU), the SLAM algorithm is used for spatiotemporal alignment and fusion to obtain the vehicle's position, vehicle posture, and high-precision real-time environmental map. The vehicle's position and posture are transmitted to the execution control unit, and the high-precision real-time environmental map is transmitted to the AI ​​adaptive decision-making system.

[0130] Edge computing nodes are used to collaboratively process the fusion environment data and computing tasks. Edge computing nodes are used for local computing optimization, transferring data processing tasks from the cloud to edge devices close to the data source to reduce the burden of cloud computing.

[0131] An AI adaptive decision-making system, which generates autonomous driving decision instructions based on fused environmental data and high-precision real-time environmental maps, using deep reinforcement learning algorithms and Markov decision process modeling;

[0132] a satellite-assisted decision-making unit, configured to optimize the autonomous driving decision instructions in combination with satellite communication data;

[0133] V2X & StarV2X communication module, used to realize vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-cloud (V2C) data exchange through 5G low-latency communication (URLLC) including 5G C and 5G NR or satellite link (Starlink), and perform real-time environmental perception;

[0134] The execution control unit is used to convert autonomous driving decision instructions into vehicle behavior, control the electronic throttle, brake and steering systems, and realize autonomous driving of the vehicle;

[0135] A path navigation trajectory calculation unit, configured to realize path navigation and trajectory generation based on the high-precision real-time environment map;

[0136] Among them, the autonomous driving system adopts an FPGA+GPU parallel computing framework, integrating FPGA and GPU parallel computing units for multimodal sensor data processing, modeling and autonomous driving decision instruction calculation.

[0137] The core technologies of the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation are as follows:

[0138] 1. Vehicle perception data

[0139] Collect data on vehicle status (position, speed, acceleration, steering angle) and external environment (road, pedestrians, obstacles, other vehicles), and pre-process it through the CAN bus and on-board computing unit (ECU).

[0140] 2. Data Fusion & Computational Decision-Making (Cloud + Local AI Computing)

[0141] The local FPGA+GPU parallel computing framework performs real-time data fusion, modeling, and decision-making calculations; important data is uploaded to the cloud (V2C) for global optimization.

[0142] 3. V2X Interaction & Broadcast (Information Transport Layer)

[0143] Vehicles communicate with other V2X devices (V2V / V2I / V2C) with low latency; 5G low-latency networks including 5GC and 5G NR and Starlink are used to ensure global coverage.

[0144] 4. Decision execution (vehicle control layer)

[0145] The driving decision data calculated by AI is transmitted to the execution control unit; the execution control unit adjusts the throttle, brake, steering wheel, gearbox, etc. to complete the autonomous driving decision.

[0146] In summary, the autonomous driving system adopts a high-performance parallel computing architecture, integrating multimodal sensor data processing, modeling and decision-making instruction calculation. The overall system delay is less than 5 milliseconds, ensuring robustness and safety in urban roads, highways and densely obstructed scenarios.

[0147] Specifically, each module in the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation is introduced as follows:

[0148] 1. Multi-sensor fusion perception system (i.e. vehicle perception system)

[0149] The vehicle perception system includes:

[0150] A laser radar unit configured to collect environmental point cloud data with a collection range greater than 250 meters, using a 64-line or equivalent high-resolution laser radar with a scanning frequency of ≥10 Hz;

[0151] a camera unit configured to collect environmental image data, wherein the resolution of the environmental image data is not less than 1920×1080 pixels and the frame rate is not less than 30 frames per second;

[0152] a millimeter-wave radar unit configured to collect environmental distance data, wherein the environmental distance data has a detection range of not less than 200 meters;

[0153] an ultrasonic sensor unit configured to collect close-range obstacle data, wherein the close-range obstacle data has a detection range of not less than 8 meters;

[0154] an infrared sensor unit configured to collect ambient thermal imaging data in a low-light environment, wherein the detection range of the ambient thermal imaging data is not less than 50 meters; and the response time of the infrared sensor unit is less than 100 milliseconds;

[0155] An ultrasonic sensor unit is configured to collect close-range obstacle data using high-power ultrasonic array technology, with a detection range of 0-8 meters and a detection accuracy of ±5 cm;

[0156] A GNSS / GPS positioning unit configured to collect vehicle position data using RTK technology, with a positioning accuracy of centimeters (2-10 cm);

[0157] Inertial Measurement Unit (IMU): Configured to collect vehicle acceleration, angular velocity, and attitude data, with a sampling frequency of ≥100 Hz, angular velocity measurement accuracy of ±0.01° / s, and acceleration measurement accuracy of ±0.01g. It supports the high-precision positioning and mapping module's positioning continuity in scenarios where GNSS signals are interrupted or blocked, and assists the dynamic occlusion compensation module's target tracking and environmental model updates.

[0158] The above configuration of the vehicle perception system enables the autonomous driving system provided by the present invention to be applicable to complex driving scenarios (urban canyons, areas with frequent occlusions), has strong applicability, and realizes multi-functional support (dynamic environment maps, path planning, occlusion compensation) and coordination with the ε-greedy strategy.

[0159] The vehicle perception system is a multi-sensor fusion perception system, such as Figure 2As shown, the multi-sensor fusion perception system obtains LiDAR point cloud data and multi-view stereo imaging and millimeter-wave radar target detection results based on the data collected by multi-modal sensors, and fuses the data collected by multi-modal sensors through data time synchronization and weight distribution to obtain multi-modal sensing data.

[0160] The LiDAR collects 3D point cloud data; the camera uses a multi-camera stereo vision module to generate a 3D semantic map of the environment through binocular disparity calculation; the millimeter-wave radar detects the radial velocity and distance of dynamic targets and tracks their trajectories using Bayesian filtering; and the IMU provides high-frequency acceleration, angular velocity, and attitude data to enhance positioning continuity in scenarios with GNSS signal interruption or occlusion, and supports occlusion area detection with dynamic occlusion compensation. Data collected by each sensor is transmitted to the ECU (on-board computing unit) via the CAN bus / Ethernet.

[0161] Multimodal sensors work together to generate high-precision, high-frequency environmental perception data (such as point clouds, images, distance, location, and posture information), supporting the generation of dynamic 3D environmental models for real-time holographic modeling. This multimodal sensing data supports the real-time holographic modeling module in generating dynamic 3D environmental models and provides the dynamic occlusion compensation module with the data required for occlusion area detection and target tracking, ensuring the system's robustness and safety in urban roads and densely obstructed environments.

[0162] The vehicle perception system further includes:

[0163] The time synchronization unit uses hardware trigger signals and PTP (Precision Time Protocol) to align the cross-modal timestamps of the raw data from the LiDAR, multi-view stereo vision module, and millimeter-wave radar, with a synchronization error of ≤100μs;

[0164] The adaptive fusion unit performs the following operations:

[0165] Dynamically adjust sensor data weights based on environmental conditions to optimize multimodal perception accuracy. For example, in rainy and foggy weather, the weight of millimeter-wave radar is increased to 0.7-0.9. In nighttime scenarios, the weight of LiDAR is increased to 0.6-0.8.

[0166] The Kalman filter fuses the data collected by multimodal sensors to construct a 360° occupancy grid map around the vehicle;

[0167] Dynamic object segmentation to identify pedestrians, vehicles, animals, road signs, etc.;

[0168] The output unit generates perception results containing the following information: semantic categories and 3D bounding boxes of static obstacles; predicted trajectories and collision risk levels of dynamic targets; and segmentation masks of drivable areas.

[0169] The autonomous driving system also includes an external data interface module for providing external supplementary data, including but not limited to traffic signal data (such as traffic light countdowns), cloud-based high-precision map data, V2X communication data (such as road construction information and surrounding vehicle trajectories), and real-time traffic information. This external supplementary data is transmitted via high-bandwidth, low-latency communications (such as V2X communications with a latency of <2 milliseconds), enhancing the environmental awareness capabilities of multimodal sensor data and supporting real-time holographic modeling and dynamic occlusion compensation.

[0170] The vehicle's perception system collects information about the vehicle's surroundings and external supplementary data, which together form the vehicle's real-time multimodal sensory data. High-precision perception data (positioning accuracy 2-10 cm, update frequency ≥10 Hz) is generated through spatiotemporal alignment and data fusion.

[0171] 2. Real-time holographic environment modeling module

[0172] like Figure 3 As shown, the real-time holographic environment modeling module provided by the present invention completes data preprocessing (filtering, denoising, calibration) of multimodal sensor data, 3D voxel grid generation, Transformer+GAN super-resolution image enhancement, deep learning environment feature extraction, holographic point cloud reconstruction, and vehicle surrounding dynamic environment modeling.

[0173] Specifically, the real-time holographic environment modeling module includes:

[0174] A data preprocessing submodule is used to preprocess multimodal sensor data, including filtering, denoising, and calibration; the multimodal sensor data includes LiDAR point cloud data, camera visual images, and millimeter-wave radar detection data;

[0175] 3D voxel grid generation submodule, which is used to convert the multimodal sensor data after data preprocessing into a 3D voxel grid model and annotate the spatial occupancy probability of the voxels;

[0176] An image enhancement submodule, configured to enhance the texture of low-resolution areas in the three-dimensional voxel grid model using a super-resolution algorithm that integrates a Transformer neural network and a generative adversarial network (GAN) to generate a high-definition holographic image;

[0177] A feature extraction submodule is used to extract multi-scale environmental features from the enhanced high-definition holographic image using a deep learning model. The multi-scale environmental features include lane lines, obstacle outlines, and dynamic target semantic labels. The multi-scale environmental features support dynamic environmental map generation, occlusion area identification, and real-time driving parameter adjustment. The multi-scale environmental feature extraction is configured to optimize feature robustness at different spatial scales to enhance environmental perception capabilities in complex driving scenarios.

[0178] a point cloud reconstruction submodule for generating a holographic point cloud model with dynamic attribute annotations using a point cloud reconstruction algorithm based on the spatial correlation between the extracted multi-scale environmental features and the voxel grid; wherein the voxel grid is constructed based on environmental perception data, the holographic point cloud model includes three-dimensional spatial information and dynamic environmental attributes, the dynamic environmental attributes including obstacle motion status and occlusion area identification, the holographic point cloud model supports real-time holographic environmental modeling, dynamic occlusion compensation, and path planning, and is configured to optimize environmental perception robustness and autonomous driving decision reliability in complex driving scenarios through dynamic attribute annotation;

[0179] The holographic imaging unit (i.e., hologram unit) is used to construct a dynamic 3D environment model of the dynamic environment around the vehicle based on the time series data of the holographic point cloud model output by the point cloud reconstruction submodule through holographic imaging technology. The dynamic 3D environment model includes a three-dimensional spatial structure and dynamic environment attributes, and the dynamic environment attributes cover road information, obstacle motion status and occlusion area identification; the holographic imaging unit cooperates with the dynamic occlusion compensation module and is configured to compensate for the perception blind spots in occlusion scenarios by analyzing the time series data and dynamic environment attributes, so as to enhance the robustness of environmental perception and the reliability of autonomous driving decisions in complex driving scenarios. The dynamic 3D environment model further supports real-time path planning and dynamic driving parameter adjustment.

[0180] The real-time holographic environment modeling module adopts 3D modeling technology and uses deep learning algorithms to perform super-resolution reconstruction of low-resolution images based on multimodal sensor data; it adopts generative adversarial networks (GANs) to improve modeling accuracy to reduce noise and artifacts; it can also combine sparse optical flow estimation and background modeling technology to improve dynamic object detection capabilities; based on a global and local feature fusion strategy, it realizes real-time perception of subtle changes in the environment.

[0181] 3. Dynamic Occlusion Compensation Module

[0182] like Figure 4 As shown, the workflow of the dynamic occlusion compensation module includes steps such as target object occlusion detection, motion prediction, Kalman filter trajectory fitting, V2X data enhancement, and occlusion compensation image restoration. The hologram unit is configured to generate a dynamic 3D environment model through holographic imaging technology with an accuracy of 0.1 meters and an update frequency of not less than 5Hz. The hologram unit cooperates with the dynamic occlusion compensation module to compensate for the perception blind spots in occlusion scenarios, with a compensation error of less than 0.3 meters. The hologram unit provides a three-dimensional environment reference for occlusion detection and image restoration through the real-time generation of a high-precision holographic environment model, thereby improving the accuracy and robustness of occlusion compensation.

[0183] Specifically, the dynamic occlusion compensation module includes:

[0184] The occlusion detection unit uses environmental segmentation and occlusion detection algorithms (such as deep learning-based data segmentation) combined with a dynamic 3D environmental model and multimodal sensor data (including lidar point clouds, camera images, millimeter-wave radar, and IMU attitude data) to identify occluded areas with a detection accuracy of ±5 cm. It collaborates with the holographic imaging unit to improve the perception accuracy of occluded areas by analyzing time series data and dynamic environmental attributes (such as obstacle motion), with a response time of <50 milliseconds.

[0185] A data compensation unit is configured to fill the occluded area based on historical holographic data or neighboring sensor data using a Bayesian estimation algorithm. Furthermore, the unit repairs visual blind spots through time series prediction and spatial interpolation to generate occlusion-compensated environmental data, supporting dynamic 3D environmental model updates for real-time holographic modeling.

[0186] The data fusion optimization unit is used to combine the dynamic 3D environmental model, the environmental data after occlusion compensation, and the external environmental data (including V2X data, cloud-based high-precision maps, and real-time traffic information) to optimize the filling results of the occluded areas and improve the filling accuracy and environmental adaptability. The optimization adopts a fusion algorithm (such as weighted fusion and deep learning) to correct the dynamic properties of the occluded areas (such as obstacle position and motion trajectory) and optimize the delay to <50 milliseconds. V2X data is used to predict the dynamic changes of the occluded areas (such as the trajectory of dynamic obstacles) to enhance the robustness of the filling results.

[0187] The occlusion detection unit includes an occlusion detection submodule, which is used to identify the occlusion area and occlusion start time of the occlusion object based on the spatial consistency analysis of the camera visual data and the LiDAR point cloud data;

[0188] The data compensation unit includes:

[0189] The Bayesian prediction submodule is used to build a probability model of the motion state of the obstruction and predict the future position distribution of the vehicle through the Bayesian inference algorithm;

[0190] A trajectory fitting submodule is used to use a Kalman filter to perform optimal trajectory estimation on the probability distribution output by the Bayesian prediction submodule to generate a continuous motion trajectory of the vehicle during the occlusion period;

[0191] The image restoration submodule is used to perform context-aware image restoration on the occluded area in the original camera image based on the corrected trajectory estimation result, thereby repairing the visual blind spot.

[0192] The dynamic occlusion compensation module provides high-precision dynamic 3D environment model support for real-time holographic modeling through accurate detection, filling and optimization of occluded areas, and enhances the reliability of autonomous driving decisions (such as path planning and obstacle avoidance), ensuring the robustness and safety of the system in urban roads, highways and densely occluded scenarios.

[0193] 4. Sensor Data Fusion Module

[0194] The sensor data fusion module includes:

[0195] a) A data preprocessing unit configured to filter and perform spatiotemporal alignment on multimodal sensor data, including point cloud data, image data, distance data, and IMU attitude data collected by the vehicle perception system, as well as dynamic 3D environment model data and occlusion-compensated environment data generated by the real-time holographic environment modeling module; denoising is performed using data filtering and preprocessing algorithms (e.g., Gaussian filtering), and data alignment is achieved in combination with timestamp synchronization and spatial coordinate calibration, with a preprocessing delay of <10 milliseconds; the IMU provides high-frequency attitude data (sampling frequency ≥100 Hz, accuracy ±0.01°), enhancing the accuracy and stability of spatiotemporal alignment;

[0196] b) A fusion computing unit configured to integrate pre-processed multimodal sensor data, dynamic 3D environment model data, and occlusion-compensated environment data through a multimodal data fusion algorithm (such as Kalman filtering and deep learning fusion) to generate fused environment data; the fused environment data has a positioning accuracy of 2-10 cm, an update frequency of ≥10 Hz, supports real-time holographic modeling and dynamic occlusion compensation, and a fusion computing delay of <50 milliseconds; and utilizes feature-level fusion of IMU attitude data, point clouds, and images to enhance the perceptual robustness of densely occluded scenes.

[0197] c) A data calibration unit is configured to calibrate fused environmental data based on external environmental data (including V2X communication data such as traffic light status, vehicle trajectory, cloud-based high-precision maps, and real-time traffic information) to improve the robustness and consistency of the fused environmental data; uses calibration algorithms (such as error correction based on external data) to correct positioning errors (calibration error ±5 cm) and environmental feature deviations, with a calibration time of <50 milliseconds; and uses V2X data to correct dynamic properties of occluded areas (such as dynamic obstacle positions) to support autonomous driving decisions (such as path planning and obstacle avoidance).

[0198] Among them, the data preprocessing unit adopts Gaussian filtering algorithm for filtering, the fusion calculation unit adopts Kalman filtering algorithm to perform data fusion, the multimodal sensor data includes point cloud data, image data and distance data collected by the vehicle perception system, and the dynamic 3D environment model data generated by the real-time holographic environment modeling module, and the processing delay of the fused environment data is less than 15 milliseconds.

[0199] In summary, the sensor data fusion module generates high-precision fused environmental data by integrating multimodal sensor data, dynamic 3D environmental models, and environmental data after occlusion compensation, providing core support for real-time holographic modeling and dynamic occlusion compensation, ensuring the robustness and safety of the system in urban roads, highways, and densely occluded scenarios.

[0200] 5. High-precision positioning and mapping module

[0201] The high-precision positioning and mapping module specifically includes:

[0202] a) A positioning data acquisition unit configured to process positioning data from the Global Navigation Satellite System (GNSS), Inertial Measurement Unit (IMU), environmental perception sensors, and connected vehicle data, and generate high-precision, high-frequency positioning information through data fusion. The positioning information meets the accuracy requirements of centimeter-level navigation and the frequency requirements of real-time positioning; the IMU data is updated at a frequency of no less than 100 Hz;

[0203] b) a map generation unit configured to generate a dynamic environment map based on a positioning and mapping algorithm, in combination with the positioning information and environmental perception data from the positioning data acquisition unit, wherein the dynamic environment map includes road information and environmental features, wherein the road information supports path planning and trajectory generation, and the environmental features support dynamic environmental adaptation. The positioning accuracy of the dynamic environment map meets centimeter-level navigation requirements, the update frequency supports dynamic driving scenarios, and the positioning accuracy of the dynamic environment map is better than 10 centimeters;

[0204] c) a map optimization unit configured to optimize the dynamic environment map by combining external supplementary data and environmental perception data to improve the map's accuracy, stability, and environmental adaptability. The external supplementary data includes cloud-based map data, real-time traffic information, and historical map data. The optimization includes updating road information, correcting environmental features, and calibrating occluded areas to support real-time environmental perception and autonomous driving decision-making, and to improve the map's robustness and consistency.

[0205] The GNSS data adopts real-time kinematic (RTK) technology, the positioning accuracy of the GNSS data is better than 2 centimeters, and the update frequency of the dynamic environment map is not less than 5Hz.

[0206] The high-precision positioning and mapping module supports dynamic occlusion compensation and is configured to identify and process occluded areas based on environmental perception data, enhancing the robustness of environmental perception and the reliability of autonomous driving decisions through occlusion detection algorithms; the dynamic environmental map is further configured to support real-time environmental adaptation, including dynamically adjusting driving parameters related to road information to respond to real-time speed limit requirements and dynamic driving scenarios.

[0207] like Figure 8 As shown, the high-precision real-time positioning and mapping module integrates SLAM and GPS to provide high-precision real-time positioning and map construction; supports multi-system GNSS (GPS, Beidou, Galileo) and SLAM data fusion; and provides the vehicle's global position, local position, posture and environmental map.

[0208] The input data for the high-precision real-time positioning and mapping module includes: GPS / GNSS data, which provides global absolute position; LiDAR data, which provides three-dimensional environmental information; camera data, which provides visual information; and IMU data, which provides short-term motion estimation. The output data of the high-precision real-time positioning and mapping module includes: vehicle position (global and local); vehicle attitude (heading, pitch, and roll); and environmental map (static and dynamic objects).

[0209] The high-precision positioning and mapping module uses data from onboard multimodal sensors, including lidar, millimeter-wave radar, cameras, ultrasonic sensors, GNSS / GPS positioning units, and inertial measurement units (IMUs). It uses positioning and mapping algorithms to achieve spatiotemporal alignment and fusion of multimodal data, generating vehicle position, vehicle posture, and a high-precision real-time map of the environment. The specific process is as follows:

[0210] 1. Multimodal data collection and collaboration:

[0211] LiDAR, millimeter-wave radar, cameras, and ultrasonic sensors provide environmental perception data, GNSS / GPS provides centimeter-level positioning data, and the IMU provides high-frequency acceleration, angular velocity, and attitude data (sampling frequency ≥ 100Hz, attitude accuracy ±0.01°). The IMU uses high-frequency attitude estimation to perform spatiotemporal alignment with the LiDAR point cloud and GNSS data, enhancing positioning continuity in situations where GNSS signals are interrupted or in complex scenarios (such as tunnels and urban canyons).

[0212] 2. Fusion of positioning and mapping algorithms:

[0213] Using positioning and mapping algorithms (such as SLAM, visual-inertial odometry, or factor graph optimization), the system fuses multimodal sensor data to generate real-time vehicle position (positioning accuracy 2-10 cm), vehicle attitude (angular accuracy ±0.01°), and a high-precision real-time environmental map (resolution ≤ 10 cm, update frequency ≥ 10 Hz). IMU-assisted dynamic environment modeling ensures map robustness in scenarios with occlusion or dense dynamic objects, and supports real-time incremental updates.

[0214] 3. Data output and application:

[0215] Vehicle position and posture are transmitted to the vehicle control module for real-time motion control and path execution. A high-precision real-time environment map is transmitted to the decision-making and planning module to support path planning, obstacle avoidance, and the dynamic occlusion compensation module's blind spot detection and target tracking. This high-precision real-time environment map, through a dynamic update mechanism (based on incremental fusion of the IMU and LiDAR), provides a high-resolution 3D environment model for dynamic occlusion compensation, ensuring robust perception and safety in complex scenarios.

[0216] The high-precision positioning and mapping module meets the positioning and mapping requirements of highly dynamic environments and occluded scenes in autonomous driving scenarios such as urban roads and highways.

[0217] 6. Edge Computing Nodes

[0218] The edge computing node includes:

[0219] a) a data processing unit configured to locally process data from the real-time holographic environment modeling module and the sensor data fusion module to support real-time environmental perception, with a computational latency of less than 50 milliseconds to meet dynamic driving requirements;

[0220] b) Task allocation unit, configured to support environmental perception and mapping tasks, low-latency, high-security collaborative communication tasks, and big data computing tasks. The scheduling delay of task allocation is less than 10 milliseconds. The scheduling delay supports real-time collaboration and ensures computing efficiency in scenarios such as urban roads and highways.

[0221] c) a data cache unit configured to cache intermediate data of the real-time holographic environment modeling module and the sensor data fusion module to improve data processing efficiency, reduce data processing latency, and support efficient data access with cache capacity;

[0222] Among them, the computing power of the edge computing node is not less than 10TOPS, the data transmission delay of the edge computing node is less than 5 milliseconds, and the cache capacity of the data cache unit is not less than 1GB.

[0223] The edge computing node adopts a high-performance edge computing architecture (such as FPGA, GPU) to provide high-performance computing capabilities to handle dynamic 3D modeling, occlusion compensation and decision-making tasks. The data transmission delay is less than 5 milliseconds, ensuring the real-time interaction and robustness of the system in urban roads, highways and densely occluded scenarios.

[0224] 7. AI Adaptive Decision-Making System

[0225] like Figure 5 As shown in the figure, the workflow of the AI ​​adaptive decision-making system consists of scene perception and classification → Markov decision process modeling → reinforcement learning decision optimization → vehicle behavior planning → AI decision execution. Data is transmitted in sequence to achieve dynamic decision optimization of the intelligent driving system in complex environments, thereby improving safety and efficiency.

[0226] 1. Scene Perception and Classification

[0227] The work content of the scene perception and classification link includes: perceiving the vehicle's surrounding environment through multi-sensor data fusion and classifying it into different driving scenarios (such as highways, urban roads, traffic light intersections, parking lots, etc.); identifying key environmental factors (traffic density, weather, road signs, pedestrians, animals and other vehicles, etc.); providing input data to the Markov decision process (MDP) to ensure that the decision-making system adaptively adjusts its strategy in different environments.

[0228] How scene perception and classification works:

[0229] Multimodal data fusion (LiDAR, camera, millimeter-wave radar, V2X): weighted Bayesian estimation is used for data fusion;

[0230] Transformer is used to extract environmental features and calculate the self-attention weight of environmental features:

[0231]

[0232] Among them, Q, K, V are input tensors, and d is the dimension;

[0233] Based on the deep learning classification model (ResNet+LSTM), ResNet extracts visual features and LSTM processes time series data for scene classification.

[0234] Scene Perception and Classification: Highly real-time, using Transformer to calculate scene features and achieve millisecond-level response; strong adaptability, dynamically adjusting classification weights to suit different driving conditions; multi-data source fusion, combining vision, radar, and V2X data, to improve classification accuracy.

[0235] 2. Markov Decision Process Modeling (MDP-MarkovDecisionProcess)

[0236] Function: Models driving decisions as a Markov decision process (MDP), providing an optimal strategy search space for reinforcement learning; calculates the optimal action strategy and transmits it to the reinforcement learning module for optimization.

[0237] Working principle: MDP is represented by a five-tuple (S, A, P, R, γ):

[0238] State space S: driving scenarios (e.g., traffic lights, lane changes, high-speed driving);

[0239] Action space A: possible driving decisions (acceleration, deceleration, braking, lane change, etc.);

[0240] State transition probability P(s'|s,a): the probability that the vehicle will transition from the current state s to the new state s' after taking action a;

[0241] Reward function R(s,a): the benefits of driving behavior (safety, comfort, fuel efficiency);

[0242] Discount factor γ: used to calculate long-term returns:

[0243] The model function is:

[0244]

[0245] Features: Probabilistically driven, able to handle uncertain scenarios (such as pedestrians suddenly crossing); highly scalable, adaptable to different driving environments (urban, highway, and severe weather); dynamic optimization, continuously adjusting strategies through reinforcement learning to optimize the driving experience.

[0246] 3. Reinforcement Learning Decision Optimization

[0247] Function: Uses deep reinforcement learning (Deep Q-Learning) to train intelligent agents to find the optimal driving strategy in the decision space of MDP modeling. Through exploration and exploitation mechanisms, it optimizes driving decisions and improves safety and efficiency.

[0248] Working principle:

[0249] (1) Q-Learning value function update:

[0250] Q(s,a)=Q(s,a)+α[R(s,a)+γmax a′ Q(s′,a′)-Q(s,a)]

[0251] Among them, Q(s,a) represents the value of taking action a in the current state, α is the learning rate, and γ is the discount factor.

[0252] (2) Use deep neural network (DNN) to approximate the Q-value function:

[0253] Q(s,a;θ)≈DNN(s,a;θ)

[0254] Convolutional neural network (CNN) is used to extract driving environment features and predict the optimal driving strategy.

[0255] (3) Experience Replay improves learning stability:

[0256] Use an experience pool (Replay Buffer) to store past decision-making experience, perform random sampling training, and reduce data correlation.

[0257] Features: Efficient decision optimization, using deep learning to accelerate Q-learning training and improve driving decision accuracy; adaptive learning, which can automatically learn the optimal strategy in different driving scenarios; avoiding local optimality: avoiding overfitting certain driving behaviors through random exploration (ε-greedy).

[0258] 4. Vehicle behavior planning

[0259] Function: Based on the results of reinforcement learning decisions, formulate specific trajectory planning and control strategies; generate safe, smooth, and energy-optimized driving paths.

[0260] Working principle:

[0261] (1) Path generation: Use cubic spline curve to calculate the optimal trajectory:

[0262] x(t)=a0+a1t+a2t 2 +a3t 3

[0263] Among them, the coefficient a i Obtained through optimization calculation.

[0264] (2) Speed ​​control, using dynamic programming to calculate the optimal acceleration:

[0265]

[0266] Where J is the acceleration change rate and w is the balance parameter.

[0267] (3) Vehicle control is performed based on MPC. The MPC objective function is:

[0268]

[0269] Among them, x d 、u d are the desired state and control input.

[0270] Features: Smooth trajectory, reducing unnecessary sudden acceleration and braking; strong predictive ability, planning vehicle behavior in advance, improving driving stability; optimal control, dynamic adjustment based on MPC, improving driving comfort.

[0271] 5. AI decision execution

[0272] The vehicle control unit (ECU) receives behavior planning data and performs operations such as steering, acceleration, and braking to achieve intelligent driving.

[0273] The AI ​​adaptive decision-making system relies on MDP+reinforcement learning to optimize driving decisions, and combines trajectory planning+predictive control to ensure the safety, comfort and efficiency of autonomous driving.

[0274] Specifically, the AI ​​adaptive decision-making system includes:

[0275] An environmental analysis unit, configured to analyze the dynamic 3D environmental model and the high-precision real-time environmental map based on a deep learning model (e.g., a convolutional neural network (CNN), a recurrent neural network (RNN), or a Transformer), perform multi-dimensional scene classification, and identify different driving scenarios, including urban roads, highways, complex intersections, and parking lots;

[0276] The Markov decision modeling module is used to build a Markov decision process model and define the state space, action space, state transition probability, and reward function. The state space includes the driving scenario, vehicle state, and environmental dynamics; the action space includes acceleration, deceleration, braking, steering, and holding strategies; the state transition probability represents the possibility of vehicle state transition based on the driving scenario and action; and the reward function evaluates driving behavior based on safety, driving efficiency, and path optimization to generate a reward value that supports real-time decision-making.

[0277] A reinforcement learning module for optimizing the Markov decision process model based on deep reinforcement learning to train the intelligent agent to generate an autonomous driving policy. The module employs a value iteration algorithm to optimize the autonomous driving policy to adapt to the dynamic 3D environment model, combines a deep neural network to extract environmental features, including obstacle locations and lane markings, and utilizes self-supervised learning to enhance policy generalization, thereby generating a policy that supports safe and efficient driving.

[0278] The vehicle behavior planning module is used to generate driving trajectories, accelerations, and obstacle avoidance paths based on the autonomous driving strategy and a dynamic 3D environment model to achieve safe and efficient vehicle control. This module uses a trajectory smoothing algorithm to generate smooth driving trajectories, an optimization algorithm to calculate accelerations to ensure driving efficiency and comfort, and a path planning algorithm to generate local obstacle avoidance paths to address dynamically blocked areas.

[0279] The AI ​​execution module is used to determine autonomous driving decision instructions based on smooth driving trajectories, optimized acceleration, and local obstacle avoidance paths.

[0280] The AI ​​adaptive decision-making system may further include:

[0281] Real-time driving behavior analysis unit, which identifies the behavior patterns of surrounding vehicles based on the AI ​​computing engine;

[0282] Abnormal driving detection unit, used to identify potentially dangerous situations such as emergency lane changes, fatigued driving, or malicious lane cutting;

[0283] A high-precision real-time prediction model is used to calculate the optimal driving trajectory and dynamically adjust it according to environmental changes.

[0284] The inter-module transmission relationship and detailed analysis in the AI ​​adaptive decision-making system:

[0285] (1→2): Scene perception and classification → Markov decision process (MDP):

[0286] Data Transmission: The scene perception and classification module processes sensor data (LiDAR, camera, millimeter-wave radar, and V2X) and identifies the driving environment using Transformer, CNN-LSTM, and Bayesian classifiers. It outputs the driving scene category C and environmental features F. The MDP model requires this information as input to determine the driving state space S and construct state transition probabilities:

[0287] P(s′|s,a)

[0288] Mathematical modeling:

[0289] (1) Driving scene classification output:

[0290]

[0291] Among them, D is the sensor data, C is the driving scene category, and the Bayesian classifier is used for prediction.

[0292] (2) State space definition:

[0293] S=\{(C,F,V,A,P)\}

[0294] Among them, C represents the current driving scene category (highway, city, tunnel, etc.); F represents environmental characteristics (traffic density, obstacles, road signs); V represents the current vehicle speed; A represents the current vehicle action (acceleration, braking, lane change); P represents the predicted future path.

[0295] (3) MDP state transition probability modeling

[0296]

[0297] The Gaussian mixture model (GMM) is used to calculate the impact of different factors on state transition.

[0298] (2→3): MDP→Reinforcement Learning Decision Optimization:

[0299] Data transmission: The MDP calculates the state transition probability P(s{\prime}|s,a) and the reward function R(s,a), and passes them to the reinforcement learning module. Reinforcement learning optimizes the driving strategy within the decision space provided by the MDP.

[0300] Mathematical modeling:

[0301] (1) Reinforcement Learning Value Function (Q-Learning Update Formula)

[0302] Q(s,a)=Q(s,a)+α[R(s,a)+γmax a′ Q(s′,a′)-Q(s,a)]

[0303] Temporal Difference (TD) error is used for updating; the discount factor \gamma affects the importance of future rewards.

[0304] (2) Deep Q-Network (DQN) approximates the Q function

[0305] Q(s,a;θ)≈f(s,a;θ)

[0306] The convolutional neural network (CNN) + Transformer structure is used to extract state features and improve decision-making accuracy.

[0307] (3)ε-Greedy Policy

[0308]

[0309] Adopt an exploration-exploitation balance strategy to avoid falling into local optimality.

[0310] Experience replay: Store past decision-making experience, perform random sampling training, and improve model stability.

[0311] Double Q-Network: Avoid Q-learning from overestimating action values ​​and improve convergence.

[0312] (3→4): Reinforcement learning decision optimization → vehicle behavior planning

[0313] Data transmission: Reinforcement learning outputs the optimal driving strategy and transmits it to the vehicle behavior planning module for trajectory generation, speed planning, and obstacle avoidance calculation.

[0314] Mathematical modeling:

[0315] (1) Trajectory planning

[0316] Use cubic spline interpolation to generate the trajectory:

[0317] x(t)=a0+a1t+a2t 2 +a3t 3

[0318] (2) Speed ​​planning

[0319] Use dynamic programming to calculate the optimal acceleration:

[0320]

[0321] Where J is the acceleration change rate and w is the balance parameter.

[0322] (3)Obstacle Avoidance

[0323] Use A* search algorithm + Dijkstra shortest path algorithm to calculate the obstacle avoidance path:

[0324] f(n)=g(n)+h(n)

[0325] Among them, g(n) represents the true cost from the starting point to the current point; h(n) represents the estimated cost from the current point to the target point.

[0326] (4→5): Vehicle behavior planning → AI decision execution

[0327] Data transmission: The trajectory, speed control, and obstacle avoidance path calculated by the vehicle behavior planning module are transmitted to the AI ​​decision execution module, and ultimately steering, acceleration, braking, and other operations are performed through the vehicle controller (ECU) and actuators.

[0328] Mathematical modeling:

[0329] (1)MPC (Model Predictive Control) calculation control instructions

[0330]

[0331] Among them, x d ,、u d Given the desired state and control input, the controller continuously predicts the optimal control command for the next T steps.

[0332] (2) PID control fine adjustment

[0333]

[0334] Where e(t) is the control error, and PID control ensures smooth execution of acceleration, braking, and steering commands.

[0335] 8. FPGA+GPU Parallel Computing Framework

[0336] like Figure 6 As shown in the figure, the workflow of the FPGA+GPU parallel computing framework includes: task allocation, low-latency data flow optimization, FPGA computing acceleration, GPU parallel computing, real-time computing task optimization, etc.

[0337] Specifically, the FPGA+GPU parallel computing framework includes:

[0338] The task allocation module is used to dynamically allocate tasks to FPGA or GPU for execution based on the real-time requirements and computing power requirements of the computing tasks, where the total computing delay meets T total =max(T FPGA ,T GPU ), and the overall system delay is constrained to <5ms;

[0339] A data flow optimization module, which uses zero-copy memory mapping and pipeline batch processing to optimize low-latency transmission of input and output data between the FPGA and GPU. It supports real-time transmission of multimodal sensor data (point cloud, image, IMU attitude), dynamic 3D environment model data, occlusion-compensated environment data, V2X data, and satellite data.

[0340] FPGAs deploy customized acceleration logic implemented in hardware description languages ​​(e.g., logic implemented in hardware description languages ​​(HDL)) to perform multimodal sensor data preprocessing (filtering, alignment), control instruction verification, and time-sensitive algorithms.

[0341] GPU, equipped with CUDA computing cores, is used for parallel processing of deep learning model inference, high-dimensional path planning, and multi-target trajectory prediction tasks;

[0342] The real-time optimization module is used to balance the computing resources of the FPGA and GPU in real time through dynamic voltage and frequency adjustment and task priority preemption. It combines V2X communication data and satellite data to dynamically adjust task priorities to support the real-time needs of occlusion compensation and path optimization.

[0343] As a high-performance heterogeneous computing architecture, the FPGA+GPU parallel computing framework supports real-time holographic modeling, dynamic occlusion compensation and autonomous driving decision-making by integrating multimodal sensor data, dynamic 3D environmental models, occlusion compensation data, V2X and satellite data, ensuring the system's computing efficiency and robustness in urban roads, highways and other scenarios with dense occlusion.

[0344] 9. V2X & StarV2X Communication Module

[0345] like Figure 7 As shown, the V2X&StarV2X communication module adopts the V2X&StarV2X vehicle network interaction architecture involving vehicle-to-vehicle communication V2V, vehicle-to-infrastructure communication V2I, vehicle-to-cloud communication V2C, low-latency 5G communication, StarV2X backup enhancement (5G-V2X&StarV2X), and traffic flow information sharing.

[0346] The V2X&StarV2X communication module includes the following units:

[0347] 1.V2X communication unit:

[0348] The system is configured to exchange data with surrounding vehicles and infrastructure via high-bandwidth, low-latency communication technologies (such as 5G NR-V2X sidelink), supporting ultra-low latency transmission. Communication latency is less than 2 milliseconds for interactions with surrounding vehicles and less than 1 millisecond for interactions with infrastructure. It provides real-time traffic data (such as traffic light countdowns, road construction information, and surrounding vehicle trajectories), supports dynamic environment map generation by the high-precision positioning and mapping module, and occlusion area prediction by the dynamic occlusion compensation module.

[0349] 2. StarV2X communication unit:

[0350] It is configured to provide data exchange via non-terrestrial communication technologies (such as low-orbit satellite communications) as a backup in the event of terrestrial communication interruptions, supporting low-latency and highly secure transmission. Communication latency is less than 5 milliseconds, and secure data transmission mechanisms (such as AES-256 or equivalent encryption protocols) are used to ensure data integrity and tamper resistance. It provides backup positioning and environmental data, enhancing the robustness of real-time holographic modeling in complex scenarios (such as remote areas and urban canyons).

[0351] 3. Communication switching unit:

[0352] The system is configured to dynamically switch between V2X and StarV2X communication units based on the communication environment (e.g., signal strength, latency threshold, and network coverage) to ensure communication continuity. Switching time is less than 50 milliseconds, meeting the overall real-time requirements of the system (e.g., low latency for dynamic occlusion compensation). Optimized switching algorithms (e.g., based on real-time network quality assessment) support communication stability in highly dynamic autonomous driving scenarios (e.g., urban roads and highways).

[0353] The V2X&StarV2X communication module enhances the dynamic 3D environmental model generation capability of the high-precision positioning and mapping module by providing real-time traffic and environmental data, and provides the low-latency data support required for occlusion area detection and target tracking for the dynamic occlusion compensation module, ensuring the robustness and safety of the system in complex scenarios.

[0354] Specifically, the V2X&StarV2X vehicle networking interaction architecture includes:

[0355] Vehicle-to-vehicle (V2V) communication module, used to broadcast vehicle status data in the 5.9 GHz band via DSRC, 5G C-V2X, or 5G NR-V2X, and receive broadcast data from surrounding vehicles;

[0356] Vehicle-to-Infrastructure (V2I) communication module, used to connect to transportation infrastructure via 5G C-V2X, 5G NR-V2X, or Wi-Fi communications, collect and process local data from transportation infrastructure in real time through edge computing nodes, and ensure data transmission error of less than 1ms based on the TSN protocol;

[0357] The vehicle-to-cloud V2C communication module is used to interact with the cloud via 5G, Wi-Fi or Starlink satellite links, upload occlusion-compensated 3D real-time holographic environment maps, multimodal sensor data, vehicle driving behavior, and receive global traffic flow optimization instructions.

[0358] The transmission mechanisms and principles of each module in the V2X & StarV2X vehicle network interaction architecture are as follows:

[0359] 1. Vehicle-to-Vehicle (V2V) Communication Mechanism

[0360] ①Data generation

[0361] The vehicle detects the surrounding environment through LiDAR, cameras, and millimeter-wave radar, generating data such as vehicle status, lane information, speed, direction, and braking status.

[0362] The data is formatted and stored in the local computing unit (ECU) and encrypted before being transmitted.

[0363] ②Data transmission

[0364] Broadcasting in the 5.9GHz frequency band using C-V2X (Cellular Vehicle-to-Everything) or DSRC (Dedicated Short Range Communication).

[0365] Data is sent via 5G or direct point-to-point (D2D) to ensure low latency (1-5ms).

[0366] ③Data processing

[0367] After the vehicle receives the V2V broadcast data, the local AI computing unit parses the data and performs trajectory prediction.

[0368] If danger is detected (such as the vehicle in front suddenly braking), the system triggers emergency braking or lane change avoidance instructions.

[0369] 2. Vehicle-to-Infrastructure (V2I) Communication Mechanism

[0370] ①Data generation

[0371] Traffic infrastructure (signal lights, cameras, road monitoring stations) collects real-time traffic information through edge computing.

[0372] Process local data through MEC (Multi-access Edge Computing) servers to reduce the burden on the cloud.

[0373] ②Data transmission

[0374] Vehicles are connected to traffic lights and road sensors via 5G C-V2X, 5G NR-V2X or Wi-Fi 6.

[0375] TSN (Time Sensitive Network) is used to ensure accurate data arrival (error <1ms).

[0376] ③Data processing

[0377] Real-time traffic information (such as traffic light countdowns and road construction data) is obtained through 5G / DSRC and input into the AIDrive computing module to optimize driving strategies:

[0378] Adjust vehicle speed according to traffic light countdown to reduce sudden braking and improve energy efficiency;

[0379] Obtain construction information ahead and automatically plan detour routes;

[0380] Combining traffic data and emergency events, it dynamically optimizes routes and speeds to ensure safe and efficient driving.

[0381] 3. Vehicle-to-Cloud Communication (V2C) Transmission Mechanism

[0382] ①Data generation

[0383] Vehicle uploaded data: real-time high-precision map updates, driving behavior, and sensor data.

[0384] The cloud server runs the AI ​​deep learning model to calculate the global optimal driving strategy.

[0385] ②Data transmission

[0386] The vehicle uploads data to the cloud via 5G (5G C or 5G NR), Wi-Fi, and satellite (Starlink).

[0387] MEC servers filter high-priority data at the edge computing layer to reduce unnecessary transmission burden.

[0388] ③Data processing

[0389] Cloud-based AI computing: predicts global traffic flow and optimizes navigation routes; calculates vehicle health status and predicts maintenance needs.

[0390] ④Data distribution

[0391] Optimized driving decisions and map data are sent to the vehicle via 5G C or 5G NR-V2X / Starlink; the vehicle updates its navigation and driving strategies to improve overall system efficiency.

[0392] 4. Low-latency 5G (5G C or 5G NR) communication (5G V2X) transmission mechanism

[0393] ①Data generation

[0394] When sensor data, Internet of Vehicles data, and cloud-based AI decision data need to be transmitted, they are first pre-processed by the edge computing unit (MEC).

[0395] ②Data transmission

[0396] 5G (5G C or 5G NR) ultra-reliable low-latency communication (URLLC) is used to ensure that critical data (such as collision warnings) has a transmission delay of less than 1ms; TSN (time-sensitive network) ensures the determinism and synchronization of transmitted data.

[0397] ③Data processing

[0398] AIDrive uses AI computing + 5G V2X to form real-time dynamic driving strategies, such as: intelligent speed limit, dynamically adjusting the speed limit based on traffic flow information to improve road traffic rate; remote driving, low-latency video & control signals support remote vehicle operation.

[0399] 5. Starlink transmission mechanism

[0400] ①Data generation

[0401] Vehicles in remote areas or on highways rely on satellite networks for V2X communications.

[0402] ②Data transmission

[0403] Vehicle → Low Earth Orbit Satellite (LEO) → Ground Base Station, with a latency of 20-30ms, which can support remote navigation and remote driving.

[0404] ③Data processing

[0405] It is suitable for scenarios such as global autonomous driving data synchronization, remote rescue, military and unmanned exploration.

[0406] 6. Traffic flow information sharing and transmission mechanism

[0407] ①Data generation

[0408] Vehicles upload their driving trajectories, traffic status, and driving intentions to the cloud; the traffic management center runs AI traffic prediction algorithms to analyze traffic conditions in the next 5-10 minutes.

[0409] ②Data transmission

[0410] Send optimized traffic management instructions (such as traffic light adjustment and intelligent speed limit) via 5G V2X / Starlink.

[0411] ③Data processing

[0412] AIDrive vehicles receive optimized driving routes, reducing congestion and improving overall driving efficiency.

[0413] The adaptive autonomous driving system integrates V2X+Starlink+AI computing to form a global seamless intelligent driving ecosystem, ensuring an efficient, safe and intelligent autonomous driving experience.

[0414] 10. Execution Control Unit

[0415] The execution control unit includes:

[0416] a) A command conversion unit is configured to convert autonomous driving decision commands into vehicle motion control signals, including throttle control signals, brake control signals, steering wheel control signals, and other control signals (such as transmission and suspension control). It supports multiple vehicle types (such as fuel vehicles and electric vehicles). It uses motion control conversion algorithms (such as kinematics-based signal mapping) to generate high-precision control signals (accuracy ±1%) with a conversion latency of <10 milliseconds. It is combined with an FPGA+GPU computing framework to accelerate command conversion and ensure real-time performance.

[0417] b) A behavioral feedback unit is configured to monitor vehicle execution status (e.g., speed, attitude, trajectory deviation) using multimodal sensor data, V2X communication data, and occlusion-compensated environmental data, and dynamically adjust autonomous driving decision instructions. This unit utilizes feedback control strategies (e.g., PID control, state estimation) with an adjustment time of <50 milliseconds and a response rate >99%, supporting dynamic adjustments in densely occluded scenarios (e.g., adjusting braking force based on occluded areas). The IMU attitude data (sampling frequency ≥100Hz, accuracy ±0.01°) enhances the accuracy of vehicle status monitoring.

[0418] c) A safety verification unit is configured to perform safety verification on motion control signals, using redundant checks and boundary condition verification to ensure that the signals meet safety requirements (such as avoiding sudden acceleration and ensuring that obstacle avoidance paths are feasible). It combines a dynamic 3D environmental model with a high-precision real-time environmental map to verify the safety of control signals in occluded areas with a verification accuracy of ±1% and a verification delay of <10 milliseconds. It also utilizes edge computing nodes to accelerate verification calculations and support real-time safety assurance.

[0419] The motion control signal includes a throttle control signal, a brake control signal and a steering wheel control signal, the feedback deviation is less than 0.1 meters, and the verification processing delay of the safety verification unit is less than 2 milliseconds.

[0420] The execution control unit supports precise control based on dynamic occlusion compensation data, optimizing autonomous driving safety and efficiency. By integrating multimodal sensor data, occlusion compensation data, V2X data, and a dynamic 3D environment model, the execution control unit generates highly accurate, safe, and reliable motion control signals and dynamically adjusts the execution state to ensure robust control and safety in urban roads, highways, and other densely occluded environments.

[0421] 11. Path Navigation Trajectory Calculation Unit

[0422] A path navigation trajectory calculation unit, configured to realize path navigation and trajectory generation based on the high-precision real-time environment map;

[0423] The path navigation trajectory calculation unit specifically includes:

[0424] a) A path planning unit configured to plan a path based on the high-precision real-time environment map using a path planning algorithm (e.g., A*, RRT) to generate a global navigation path with centimeter-level accuracy. The navigation path has a planning accuracy of 2-10 cm and a planning time of <50 milliseconds, adapting to dynamic driving scenarios such as urban roads and highways. The unit can also use occlusion compensation data to predict occluded areas and dynamic obstacles to optimize path avoidance strategies.

[0425] b) a trajectory calculation unit configured to calculate the vehicle's driving trajectory based on the navigation path, with a trajectory accuracy of ±5 cm and a calculation delay of <50 milliseconds, meeting real-time driving requirements; the IMU can provide high-frequency attitude data (sampling frequency ≥100 Hz, accuracy ±0.01°) to enhance the continuity and stability of trajectory calculation;

[0426] c) a trajectory optimization unit configured to optimize the driving trajectory by combining the external supplementary data and the occlusion-compensated environmental data to improve the smoothness and safety of the driving trajectory;

[0427] The path planning unit adopts a path planning algorithm to perform path planning, and the optimization processing delay of the trajectory optimization unit is less than 5 milliseconds.

[0428] The path navigation trajectory calculation unit supports real-time holographic modeling and dynamic occlusion compensation by integrating high-precision real-time environmental maps, multimodal sensor data and external environmental data, ensuring the robustness and safety of the system's path planning and trajectory generation in urban roads, highways and densely occluded scenarios.

[0429] 12. Satellite-Assisted Decision-Making Unit

[0430] The satellite-assisted decision-making unit comprises:

[0431] a) A satellite data receiving unit configured to receive high-precision positioning data (e.g., satellite differential positioning, with an accuracy of 2-10 cm) and wide-area environmental data (e.g., regional traffic information, weather data, and occlusion area predictions) via non-terrestrial high-bandwidth communication technologies (e.g., low-orbit satellite communications) to support real-time decision-making needs; employing secure data transmission mechanisms (e.g., AES-256 encryption) to ensure data integrity and tamper resistance; the wide-area environmental data complements multimodal sensor data to support dynamic occlusion compensation;

[0432] b) A decision optimization computing unit is configured to optimize autonomous driving decision instructions using decision optimization algorithms (such as dynamic path replanning and risk assessment) based on high-precision positioning data and wide-area environmental data received by the satellite data receiving unit, combined with fused environmental data, a dynamic 3D environmental model, and a high-precision real-time environmental map. The optimized decision instructions improve path planning accuracy (±5 cm) and driving safety (such as avoiding dynamic obstacles in obstructed areas), with a computational latency of <50 milliseconds, meeting dynamic driving requirements. Edge computing nodes are used to accelerate optimization calculations and support real-time decision making.

[0433] c) A data synchronization unit configured to synchronize data received by a satellite data receiving unit with fused environmental data, V2X communication data, and IMU attitude data, using timestamp alignment and spatial calibration mechanisms to enhance the real-time and consistency of autonomous driving decision instructions; support decision-making robustness in densely occluded scenes and remote areas; and enhance the spatiotemporal consistency of data synchronization using the IMU attitude data (sampling frequency ≥ 100 Hz, accuracy ±0.01°).

[0434] The satellite data receiving unit adopts Ka-band communication, and the optimization processing delay of the decision optimization calculation unit is less than 5 milliseconds.

[0435] The satellite-assisted decision-making unit integrates satellite data, fuses environmental data and V2X data, supports dynamic occlusion compensation (providing a wide-area perspective of the occluded area) and real-time path optimization (correcting the global path), ensuring the system's decision-making accuracy and safety in urban roads, highways, densely obstructed scenarios and remote areas.

[0436] Example 2

[0437] The present invention provides an autonomous driving method based on real-time holographic environment modeling and dynamic occlusion compensation. The method realizes precise autonomous driving of a vehicle in complex scenarios through real-time multimodal data processing, dynamic environment modeling, and intelligent decision-making. The method includes the following steps:

[0438] Based on the vehicle's real-time multimodal sensor data, a dynamic 3D environment model is constructed using holographic imaging technology;

[0439] Based on a dynamic 3D environmental model, the system uses Bayesian reasoning and Kalman filtering algorithms, combined with multimodal sensor data and vehicle-to-vehicle communication data, to predict occluded areas and the motion trajectory of obstructing objects. It also uses V2X vehicle-to-everything (V2X) data to enhance prediction accuracy, repair visual blind spots, and obtain occlusion-compensated environmental data.

[0440] Performing spatiotemporal alignment and fusion processing on the multimodal sensing data, the dynamic 3D environment model constructed by the real-time holographic environment modeling module, and the occlusion-compensated environment data obtained by the dynamic occlusion compensation module to obtain fused environment data;

[0441] Based on the fusion of environmental data and high-precision environmental maps, deep reinforcement learning algorithms and Markov decision process modeling are used to generate autonomous driving decision instructions;

[0442] optimizing the autonomous driving decision instructions in conjunction with satellite communication data;

[0443] Transmitting autonomous driving decision instructions to the execution control unit through the V2X & StarV2X communication module;

[0444] The execution control unit converts the autonomous driving decision instructions into vehicle behavior, controls the electronic throttle, brake and steering systems, and realizes the vehicle's autonomous driving;

[0445] Among them, the autonomous driving method adopts the FPGA+GPU parallel computing framework to process multimodal sensor data, modeling and autonomous driving decision instruction calculation, ensuring that the overall system delay is less than 5ms.

[0446] The following are specific operating steps of the autonomous driving method based on real-time holographic environment modeling and dynamic occlusion compensation in the embodiment:

[0447] 1. Real-time holographic environment modeling

[0448] Operation steps: Start the vehicle's multimodal sensors (including LiDAR, cameras, radar and hologram) to collect environmental data in real time; input the collected multimodal sensor data into the FPGA+GPU parallel computing framework, run the pre-installed super-resolution image reconstruction algorithm (such as SRGAN based on deep convolutional network), and combine it with the light field data of the hologram to build a real-time 3D holographic environment model; through real-time data stream processing, ensure that the model update frequency is not less than 20Hz.

[0449] Output: Generates a real-time 3D holographic environment map, including dynamic spatial distribution information of roads, obstacles and dynamic objects.

[0450] 2. Occlusion Compensation and Prediction

[0451] Operation steps: Based on the real-time 3D holographic environment model generated in step 1, load the Bayesian reasoning and Kalman filtering algorithm into the FPGA+GPU framework to predict the motion trajectory of obstructed areas (such as pedestrians, animals, and vehicles blocked by the vehicle in front); receive shared data (such as position and speed) from surrounding vehicles and infrastructure through the V2X vehicle network module (supporting 5G NR or 5G C-V2X or DSRC), and integrate it into the prediction model to improve trajectory estimation accuracy; use the prediction results to repair visual blind spots and update the real-time 3D holographic environment map.

[0452] Output result: A real-time 3D holographic environment map after dynamic occlusion compensation is obtained, eliminating perception blind spots and maintaining dynamic consistency of the map.

[0453] 3. Generate autonomous driving strategy

[0454] Operation steps: Input the real-time 3D holographic environment map after dynamic occlusion compensation into the AI ​​decision-making module, run deep reinforcement learning algorithms such as (DQN (retrieval required) or PPO) and Markov decision process (MDP) model on the GPU, and generate real-time autonomous driving strategies based on predefined reward functions (taking into account safety, efficiency, etc.); through self-supervised learning, use real-time driving data and feedback to train the strategies online and optimize decision results; generate specific vehicle control instructions (such as acceleration, braking, and steering angles).

[0455] Output result: Obtain the optimal vehicle control instructions adapted to the current scenario, ensuring real-time and decision-making accuracy.

[0456] 4. Command transmission

[0457] Operation steps: Through the V2X & StarV2X vehicle network interaction architecture (integrated 5G NR and Starlink satellite communications), the vehicle control instructions generated in step 3 are transmitted to the vehicle's execution control unit in a low-latency manner (delay <5ms); if the main link (such as 5G) is interrupted, switch to the StarV2X backup link to ensure transmission continuity.

[0458] Output result: Control instructions are reliably transmitted to the execution unit.

[0459] 5. Execute vehicle behavior

[0460] Operation steps: After the execution control unit receives the control command, it converts the command into specific signals of electronic throttle opening, braking force and steering angle through the CAN bus or Ethernet interface; drives the vehicle actuators (such as electronic throttle module, braking system, steering motor) to realize the vehicle's automatic driving behavior.

[0461] Output result: The vehicle completes acceleration, deceleration or steering and other actions in real time according to the instructions to achieve the expected driving goal.

[0462] 6. Computing framework support

[0463] Operation steps: In the entire control process, configure the FPGA+GPU parallel computing framework, where the FPGA is responsible for real-time preprocessing and fusion of multimodal sensor data (such as point cloud denoising, image correction, and light field decoding), and the GPU is responsible for complex model calculations (such as real-time holographic modeling and reinforcement learning inference), ensuring that the total delay of data processing and strategy generation is less than 5ms.

[0464] Output results: Efficient and real-time computing support to ensure the system's dynamic response capabilities.

[0465] This embodiment significantly improves the perception accuracy and decision-making ability of the autonomous driving system in occlusion scenarios through real-time holographic environment modeling (integration of super-resolution algorithm and hologram data), dynamic occlusion compensation (Bayesian reasoning + Kalman filtering + V2X fusion), and real-time strategy optimization of deep reinforcement learning and MDP; the FPGA+GPU framework and V2X&StarV2X architecture ensure real-time performance (update frequency ≥ 20Hz, delay < 5ms) and reliability.

[0466] Example 3

[0467] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by at least one processor of the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation as described above, the at least one processor implements the above-mentioned autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation, thereby realizing precise autonomous driving of the vehicle in complex scenarios.

[0468] This storage medium supports high real-time and robustness for L4 / L5 autonomous driving.

[0469] The present invention also provides a computer program product, including a computer program. When the computer program is executed by at least one processor of the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation as described above, the at least one processor implements the above autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation, thereby realizing precise autonomous driving of the vehicle in complex scenarios.

[0470] Example 4

[0471] An electronic device, characterized in that the electronic device includes: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the automatic driving method based on real-time holographic modeling and dynamic occlusion compensation.

[0472] Example 5

[0473] The present invention also provides an autonomous driving vehicle equipped with the above-mentioned autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation, which implements the autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation to perform autonomous driving control of the vehicle.

[0474] The autonomous driving vehicle may also be installed with the electronic equipment described in Example 4 to realize the autonomous driving function.

[0475] The autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation integrates holographic imaging technology, multimodal sensor data fusion, dynamic occlusion compensation, edge computing, vehicle-to-vehicle communication, and AI adaptive decision-making technology, making the vehicle suitable for a variety of complex driving scenarios such as urban roads, highways, and tunnels, providing high-precision environmental perception and intelligent driving control capabilities; greatly improving the vehicle's autonomous driving level and achieving efficient, safe, and intelligent L4 / L5 autonomous driving.

[0476] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. An autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation, characterized in that: include: A real-time holographic environment modeling module, which uses holographic imaging technology to build a dynamic 3D environment model based on the vehicle's real-time multimodal sensor data; The dynamic occlusion compensation module uses Bayesian reasoning and Kalman filtering algorithms based on a dynamic 3D environment model, combined with multimodal sensor data and vehicle-to-vehicle communication data, to predict occluded areas and the motion trajectory of occluders. It also uses V2X vehicle-to-everything (V2X) data to enhance prediction accuracy, repair visual blind spots, and generate occlusion-compensated environmental data. a sensor data fusion module for performing spatiotemporal alignment and fusion processing on the multimodal sensor data, the dynamic 3D environment model constructed by the real-time holographic environment modeling module, and the occlusion-compensated environment data obtained by the dynamic occlusion compensation module to obtain fused environment data; High-precision positioning and mapping module, used to generate high-precision real-time environmental maps; Edge computing nodes, used to collaboratively process the fusion environment data and computing tasks; An AI adaptive decision-making system, which generates autonomous driving decision instructions based on fused environmental data and high-precision real-time environmental maps, using deep reinforcement learning algorithms and Markov decision process modeling; a satellite-assisted decision-making unit, configured to optimize the autonomous driving decision instructions in combination with satellite communication data; V2X & StarV2X communication module, used to achieve vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-cloud (V2C) data exchange and real-time environmental awareness through 5G low-latency communications including 5G C and 5G NR or satellite links; The execution control unit is used to convert autonomous driving decision instructions into vehicle behavior, control the electronic throttle, brake and steering systems, and realize autonomous driving of the vehicle; Among them, the autonomous driving system adopts an FPGA+GPU parallel computing framework, integrating FPGA and GPU parallel computing units for multimodal sensor data processing, modeling and autonomous driving decision instruction calculation.

2. The autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The autonomous driving system also includes a vehicle perception system and an external data interface module: The vehicle perception system includes a multimodal sensor carried by the vehicle for collecting information about the vehicle's surrounding environment; the multimodal sensor includes a lidar, a millimeter-wave radar, a camera, an ultrasonic sensor, a GNSS / GPS positioning unit, and an inertial measurement unit (IMU); The external data interface module is used to provide external supplementary data, which includes traffic signal data, cloud-based high-precision map data, V2X communication data, and real-time traffic information; The vehicle's surrounding environment information and external supplementary data constitute the vehicle's real-time multimodal sensing data, and high-precision perception data is generated through spatiotemporal alignment and data fusion.

3. The automatic driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 2, characterized in that: The multimodal sensor comprises: A laser radar unit configured to collect environmental point cloud data with a collection range greater than 250 meters, using a 64-line or equivalent high-resolution laser radar with a scanning frequency of ≥10 Hz; a millimeter-wave radar unit configured to collect environmental distance data, wherein the environmental distance data has a detection range of not less than 200 meters; a camera unit configured to collect environmental image data, wherein the resolution of the environmental image data is not less than 1920×1080 pixels and the frame rate is not less than 30 frames per second; An ultrasonic sensor unit is configured to collect close-range obstacle data using high-power ultrasonic array technology, with a detection range of 0-8 meters and a detection accuracy of ±5 cm; A GNSS / GPS positioning unit configured to collect vehicle position data using RTK technology, with a positioning accuracy of centimeters (2-10 cm); The inertial measurement unit (IMU) is configured to collect vehicle acceleration, angular velocity, and attitude data with a sampling frequency of ≥100 Hz, an angular velocity measurement accuracy of ±0.01° / s, and an acceleration measurement accuracy of ±0.01g. It supports the high-precision positioning and mapping module's positioning continuity in scenarios where the GNSS signal is interrupted or blocked, and assists the dynamic occlusion compensation module's target tracking and environmental model updates.

4. The autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The autonomous driving system further includes: A path navigation trajectory calculation unit, configured to realize path navigation and trajectory generation based on the high-precision real-time environment map; The path navigation trajectory calculation unit includes: A path planning unit is configured to perform path planning based on the high-precision real-time environment map using a path planning algorithm to generate a global navigation path with path accuracy supporting centimeter-level navigation requirements; a trajectory calculation unit configured to calculate a driving trajectory of the vehicle based on the global navigation path, wherein the trajectory calculation delay meets real-time driving requirements; A trajectory optimization unit is configured to optimize the driving trajectory by combining the external supplementary data and the occlusion-compensated environmental data to improve trajectory continuity and collision safety, and the trajectory optimization delay supports dynamic scenarios.

5. The autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The real-time holographic environment modeling module includes: A data preprocessing submodule is used to preprocess multimodal sensor data, including filtering, denoising, and calibration; the multimodal sensor data includes LiDAR point cloud data, camera visual images, and millimeter-wave radar detection data; 3D voxel grid generation submodule, which is used to convert the multimodal sensor data after data preprocessing into a 3D voxel grid model and annotate the spatial occupancy probability of the voxels; An image enhancement submodule, configured to enhance the texture of low-resolution areas in the three-dimensional voxel grid model using a super-resolution algorithm that integrates a Transformer neural network and a generative adversarial network (GAN) to generate a high-definition holographic image; A feature extraction submodule is used to extract multi-scale environmental features from the enhanced high-definition holographic image using a deep learning model. The multi-scale environmental features include lane lines, obstacle outlines, and dynamic target semantic labels. The multi-scale environmental features support dynamic environmental map generation, occlusion area recognition, and real-time driving parameter adjustment. The point cloud reconstruction submodule is used to generate a holographic point cloud model with dynamic attribute annotations through a point cloud reconstruction algorithm based on the spatial correlation between the extracted multi-scale environmental features and the voxel grid; The holographic imaging unit is used to construct a dynamic 3D environmental model of the dynamic environment around the vehicle based on the time series data of the holographic point cloud model output by the point cloud reconstruction submodule through holographic imaging technology. The dynamic 3D environmental model includes a three-dimensional spatial structure and dynamic environmental attributes. The dynamic environmental attributes cover road information, obstacle motion status and occlusion area identification. The holographic imaging unit cooperates with the dynamic occlusion compensation module and is configured to compensate for the perception blind spots in occlusion scenarios by analyzing the time series data and dynamic environmental attributes.

6. The automatic driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 5, characterized in that: The dynamic occlusion compensation module includes: an occlusion detection unit, configured to identify occlusion areas in the dynamic 3D environment model using a data segmentation algorithm and collaborate with the holographic imaging unit to improve the accuracy of occlusion area perception; a data compensation unit, configured to fill the occluded area based on historical holographic data or adjacent sensor data using a Bayesian estimation algorithm; The data fusion optimization unit is used to combine the dynamic 3D environment model and external supplementary data to optimize the filling result of the occluded area.

7. The autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The sensor data fusion module includes: a data preprocessing unit configured to filter and align the multimodal sensor data; a fusion computing unit configured to perform data fusion and generate fusion environment data; a data calibration unit configured to calibrate the fused environment data based on external supplementary data to improve the robustness of the fused environment data; Among them, the data preprocessing unit adopts Gaussian filtering algorithm for filtering, the fusion calculation unit adopts Kalman filtering algorithm to perform data fusion, and the multimodal sensor data includes point cloud data, image data and distance data collected by the vehicle perception system, and dynamic 3D environment model data generated by the real-time holographic environment modeling module.

8. The autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The high-precision positioning and mapping module is used to generate a high-precision real-time environment map, specifically including: The high-precision positioning and mapping module uses data collected by the vehicle's onboard lidar, millimeter-wave radar, camera, ultrasonic sensor, GNSS / GPS positioning unit and inertial measurement unit (IMU) to perform spatiotemporal alignment and fusion using the SLAM algorithm to obtain the vehicle's position, posture and high-precision real-time environmental map. The vehicle's position and posture are transmitted to the execution control unit, and the high-precision real-time environmental map is transmitted to the AI ​​adaptive decision-making system.

9. The autonomous driving system based on real-time holographic environment modeling and dynamic occlusion compensation according to claim 8, characterized in that: The high-precision positioning and mapping module includes: A positioning data acquisition unit configured to process positioning data from Global Navigation Satellite System (GNSS) data, Inertial Measurement Unit (IMU) data, environmental perception sensors, and Internet of Vehicles data, and generate high-precision, high-frequency positioning information through data fusion. The positioning information meets the accuracy requirements of centimeter-level navigation and the frequency requirements of real-time positioning; a map generation unit configured to generate a dynamic environment map based on a positioning and mapping algorithm, in combination with the positioning information and environmental perception data from the positioning data acquisition unit, wherein the dynamic environment map includes road information and environmental features, wherein the road information supports path planning and trajectory generation, and the environmental features support dynamic environment adaptation, wherein the positioning accuracy of the dynamic environment map meets centimeter-level navigation requirements, and the update frequency supports dynamic driving scenarios; a map optimization unit configured to optimize the dynamic environment map by combining external supplementary data and environmental perception data to improve the map's accuracy, stability, and environmental adaptability, wherein the external supplementary data includes cloud-based map data, real-time traffic information, and historical map data. The optimization includes updating road information, correcting environmental features, and calibrating occluded areas to support real-time environmental perception and autonomous driving decision-making; The high-precision positioning and mapping module supports dynamic occlusion compensation and is configured to identify and process occluded areas based on environmental perception data, enhancing the robustness of environmental perception and the reliability of autonomous driving decisions through occlusion detection algorithms; the dynamic environmental map is further configured to support real-time environmental adaptation, including dynamically adjusting driving parameters related to road information to respond to real-time speed limit requirements and dynamic driving scenarios.

10. The automatic driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The edge computing node includes: a data processing unit configured to locally process the perception data of the real-time holographic environment modeling module and the sensor data fusion module to support real-time environment perception and calculate delays to meet dynamic driving requirements; A task allocation unit, configured to schedule perception and mapping tasks, high-reliability communication tasks, and high-computation tasks, with scheduling delays supporting real-time collaboration; Data cache unit, configured to cache-aware and fuse data to reduce data processing latency, with cache capacity supporting efficient data access; Among them, the edge computing nodes provide high-performance computing capabilities, support dynamic occlusion compensation and decision-making tasks, and the data transmission delay meets real-time interaction requirements.

11. The automatic driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The AI ​​adaptive decision-making system includes: An environmental analysis unit, configured to analyze the dynamic 3D environmental model and the high-precision real-time environmental map based on a deep learning model, perform multi-dimensional scene classification, and identify different driving scenarios, including urban roads, highways, complex intersections, and parking lots; the deep learning model includes a convolutional neural network (CNN) and a recurrent neural network (RNN); The Markov decision modeling module is used to build a Markov decision process model and define the state space, action space, state transition probability, and reward function. The state space includes the driving scenario, vehicle state, and environmental dynamics; the action space includes acceleration, deceleration, steering, braking, and holding strategies; the state transition probability represents the possibility of vehicle state transition based on the driving scenario and action; and the reward function evaluates driving behavior based on safety, driving efficiency, and path optimization to generate a reward value that supports real-time decision-making. A reinforcement learning module for optimizing the Markov decision process model based on deep reinforcement learning to train the intelligent agent to generate an autonomous driving policy. The module employs a value iteration algorithm to optimize the autonomous driving policy to adapt to the dynamic 3D environment model, combines a deep neural network to extract environmental features, including obstacle locations and lane markings, and utilizes self-supervised learning to enhance policy generalization, thereby generating a policy that supports safe and efficient driving. The vehicle behavior planning module is used to generate driving trajectories, accelerations, and obstacle avoidance paths based on the autonomous driving strategy and a dynamic 3D environment model to achieve safe and efficient vehicle control. This module uses a trajectory smoothing algorithm to generate smooth driving trajectories, an optimization algorithm to calculate accelerations to ensure driving efficiency and comfort, and a path planning algorithm to generate local obstacle avoidance paths to address dynamically blocked areas. The AI ​​execution module is used to determine autonomous driving decision instructions based on smooth driving trajectories, optimized acceleration, and local obstacle avoidance paths.

12. The automatic driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The satellite-assisted decision-making unit comprises: A satellite data receiving unit configured to receive high-precision positioning data or wide-area environmental data, with data reception latency supporting real-time decision-making requirements; a decision optimization unit configured to optimize the autonomous driving decision instructions based on the data received by the satellite data receiving unit to improve path planning accuracy or driving safety, and optimize the calculation delay of the autonomous driving decision instructions to meet dynamic driving requirements; a data synchronization unit configured to synchronize the data received by the satellite data receiving unit with the fused environment data to enhance the real-time and consistency of the autonomous driving decision instructions; The satellite data receiving unit adopts high-bandwidth satellite communication technology; the satellite-assisted decision-making unit supports dynamic occlusion compensation and real-time path optimization.

13. The autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The V2X&StarV2X communication module includes: The V2X communication unit is used to exchange data with surrounding vehicles or infrastructure through 5G C or 5G NR-V2X sidelink communication, supporting ultra-low latency data transmission; The StarV2X communication unit provides data interaction via satellite communications, serving as a backup in the event of ground communication interruptions and supporting low-latency and high-security transmission; a communication switching unit, configured to dynamically switch between the V2X communication unit and the StarV2X communication unit according to a communication environment to ensure communication continuity; Among them, the communication delay between the V2X communication unit and surrounding vehicles is less than 2 milliseconds, and the transmission error with the infrastructure is less than 1 millisecond; the communication delay of the StarV2X communication unit is less than 5 milliseconds, and high-level data encryption is adopted; the switching time of the communication switching unit is less than the overall system delay requirement.

14. The automatic driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The FPGA+GPU parallel computing framework includes: The task allocation module is used to dynamically allocate tasks to FPGA or GPU for execution based on the real-time requirements and computing power requirements of the computing tasks, where the total computing delay meets T total =max(T FPGA ,T GPU ), and the overall system delay is constrained to <5ms; The data flow optimization module is used to optimize the low-latency transmission of input and output data between the FPGA and GPU using zero-copy memory mapping and pipeline batch processing technology; FPGA, deploying customized acceleration logic implemented in hardware description language to perform multimodal sensor data preprocessing, control instruction verification, and time-sensitive algorithms; GPU, equipped with CUDA computing cores, is used for parallel processing of deep learning model inference, high-dimensional path planning, and multi-target trajectory prediction tasks; The real-time optimization module is used to perform real-time load balancing of FPGA and GPU computing resources through dynamic voltage and frequency adjustment and task priority preemption mechanism.

15. The automatic driving system based on real-time holographic modeling and dynamic occlusion compensation according to claim 1, characterized in that: The execution control unit includes: an instruction conversion unit, configured to convert the autonomous driving decision instruction into a motion control signal for the vehicle; A behavior feedback unit, configured to monitor the vehicle's execution status and adjust the autonomous driving decision instructions; A safety verification unit, configured to perform safety verification on the motion control signal; The motion control signal includes a throttle control signal, a brake control signal and a steering wheel control signal.

16. An autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation, applied to the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation according to any one of claims 1 to 15, characterized in that: The following steps are involved: Based on the vehicle's real-time multimodal sensor data, a dynamic 3D environment model is constructed using holographic imaging technology; Based on a dynamic 3D environmental model, the system uses Bayesian reasoning and Kalman filtering algorithms, combined with multimodal sensor data and vehicle-to-vehicle communication data, to predict occluded areas and the motion trajectory of obstructing objects. It also uses V2X vehicle-to-everything (V2X) data to enhance prediction accuracy, repair visual blind spots, and obtain occlusion-compensated environmental data. Perform spatiotemporal alignment and fusion processing on multimodal sensor data, dynamic 3D environment model and occlusion-compensated environment data to obtain fused environment data; Based on the fusion of environmental data and high-precision environmental maps, deep reinforcement learning algorithms and Markov decision process modeling are used to generate autonomous driving decision instructions; Optimize autonomous driving decision instructions by combining satellite communication data; Transmitting autonomous driving decision instructions to the execution control unit through the V2X & StarV2X communication module; The execution control unit converts the autonomous driving decision instructions into vehicle behavior, controls the electronic throttle, brake and steering systems, and realizes the vehicle's autonomous driving; Among them, the autonomous driving method adopts the FPGA+GPU parallel computing framework to process multimodal sensor data, modeling and autonomous driving decision instruction calculation, ensuring that the overall system delay is less than 5ms.

17. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by at least one processor of the autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation as described in any one of claims 1 to 15, the at least one processor implements the autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation as described in claim 16.

18. An electronic device, characterized in that: The electronic device includes: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to execute the autonomous driving method based on real-time holographic modeling and dynamic occlusion compensation as described in claim 16.

19. An autonomous driving vehicle, characterized in that: An autonomous driving system based on real-time holographic modeling and dynamic occlusion compensation as described in any one of claims 1-15.

Images