Intelligent automobile automatic driving method and system based on synergy of heaven and earth

Abstract

The application discloses a method and system for intelligent vehicle automatic driving in the integration of heaven and earth, and the implementation steps include: starting the vehicle terminal, calibrating the vehicle positioning information, and accessing the cloud service center through the communication double link; starting the automatic driving main system and the twin system, dynamically switching the automatic driving mode according to the vehicle state to the primary mode, the intermediate mode and the advanced mode; if it is the primary mode, starting the basic automatic driving strategy and method in the primary mode; if it is the intermediate mode, starting the double-system automatic driving strategy and method in the intermediate mode; if it is the advanced mode, starting the vehicle-cloud collaborative automatic driving strategy and method in the advanced mode; and starting the emergency driving method when the main system is abnormally failed. The application realizes the all-around automatic driving in the integration of heaven and earth in the complex driving environment, enhances the reliability of the system, and realizes efficient and safe vehicle control through the multi-level automatic driving mode.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to an intelligent vehicle autonomous driving method and system that integrates space, ground, and autonomous driving. Background Technology

[0002] With the rapid development of information technology and artificial intelligence, autonomous driving technology has become a research hotspot and a major development direction in fields such as vehicle engineering and transportation engineering. Intelligent vehicles integrate multiple sensors and advanced algorithms to achieve perception, decision-making, and control processes, thereby enabling autonomous driving. In recent years, end-to-end autonomous driving models, as an innovative technological approach, aim to directly integrate perception, decision-making, and control processes into a unified model through methods such as deep learning. This simplifies the complex modular design of traditional autonomous driving systems, improves system response speed and overall performance, and has already been applied in single-vehicle intelligence.

[0003] While single-vehicle intelligence is relatively mature and has achieved some applications, single-vehicle sensors are easily limited by factors such as field of view, installation location, and weather conditions, making it difficult to comprehensively and accurately perceive the dynamic environment around the vehicle. Therefore, vehicle-to-infrastructure (V2I) cooperative autonomous driving technology has gained increasing attention and development. However, roadside equipment is often costly and requires road modifications, resulting in slow construction speed and low coverage. In contrast, low-Earth orbit (LEO) satellites offer wide coverage, low cost, and rapid deployment. They can provide real-time, high-precision positioning and navigation services for ground vehicles and support large-scale data transmission and processing, providing a new foundation for V2I-cloud cooperative autonomous driving technology.

[0004] The integrated space-ground cooperative autonomous driving system significantly improves the comprehensiveness and accuracy of environmental perception and enhances the system's robustness and responsiveness by enabling information sharing and collaborative work between ground vehicles and low-Earth orbit (LEO) satellites. LEO satellites not only provide high-precision vehicle positioning and navigation information but also monitor traffic flow, road conditions, and potential hazards in real time through wide-area coverage, feeding this information back to ground vehicles to assist them in making more accurate route planning and decisions. Furthermore, the high bandwidth and low latency of LEO satellite networks enable the real-time transmission and processing of large-scale vehicle data, thereby supporting more complex and efficient autonomous driving algorithms.

[0005] Against this backdrop, this invention designs a space-ground integrated collaborative intelligent vehicle autonomous driving method and system. This system not only fully leverages the high-precision positioning and wide-area communication advantages of low-orbit satellites to enhance the environmental perception and decision-making capabilities of the autonomous driving system, but also strengthens the overall system's flexibility and robustness, ensuring safe and efficient autonomous driving in various complex and dynamic traffic environments. The research and application of this type of model has broad market prospects and significant scientific research value, and will drive the development of intelligent transportation systems towards greater intelligence and collaboration. Summary of the Invention

[0006] This invention proposes a space-ground integrated collaborative intelligent vehicle autonomous driving method and system, achieving efficient and safe vehicle control through multi-level autonomous driving modes. The method utilizes dual links of mobile and satellite communication to access the cloud service center, ensuring reliable and low-latency communication. High-precision vehicle positioning and navigation are achieved via low-Earth orbit satellites. Autonomous driving is implemented through a main system and a twin system, switching between primary, intermediate, and advanced driving modes based on vehicle status. Furthermore, an emergency driving method is designed for main system failures. Overall, this invention achieves comprehensive space-ground integrated collaborative autonomous driving in complex driving environments, enhancing system redundancy and reliability, and significantly improving the safety and efficiency of autonomous driving. To achieve the above objectives, this invention adopts the following technical solution:

[0007] A space-ground integrated collaborative intelligent vehicle autonomous driving method, the method comprising:

[0008] S1. The vehicle terminal starts up, calibrates the vehicle positioning information, and connects to the cloud service center through dual communication links;

[0009] S2. Activate the main autonomous driving system and twin system, and dynamically switch autonomous driving modes according to the vehicle status;

[0010] S3. If the autonomous driving mode is the primary mode, then the basic autonomous driving strategy and method under the primary mode will be activated.

[0011] S4. If the autonomous driving mode is intermediate mode, then activate the dual-system autonomous driving strategy and method in intermediate mode.

[0012] S5. If the autonomous driving mode is advanced mode, then activate the vehicle-cloud collaborative autonomous driving strategy and method in advanced mode.

[0013] S6. If the main system fails abnormally, activate the emergency driving method.

[0014] To further explain, step S1 specifically includes:

[0015] S11. The vehicle terminal starts, the on-board components perform self-test, the on-board sensors are initialized and calibrated;

[0016] S12. The vehicle terminal opens the network interface, detects and connects to the mobile communication network and the satellite communication network, sends a data packet for the first time to determine the reachability of the dual communication links, connects to the cloud service center through the dual links, and calibrates the vehicle positioning information at the same time.

[0017] S13. The vehicle terminal and cloud service center periodically send data packets to determine the reachability and latency of the two communication links. The link with the lowest latency is selected as the primary link for data transmission, while the other link is the communication backup link. When the primary link is unreachable or the latency is too high, the roles of the communication backup link and the primary link are swapped.

[0018] To further explain, step S2 specifically involves:

[0019] S21. Read the vehicle's autonomous driving configuration file, start the main autonomous driving system, run the main system's self-test data to ensure the availability and accuracy of basic autonomous driving, and enter the primary mode;

[0020] S22. After ensuring the stable operation of the main autonomous driving system, start the twin system, load the end-to-end network, and run the twin system self-test data. If the self-test passes, enter the intermediate mode and achieve autonomous driving through the dual systems. Otherwise, stay in the primary mode and rely solely on the main system to achieve autonomous driving.

[0021] S23. When the autonomous driving system is in intermediate mode, it attempts to obtain vehicle-cloud collaborative information through dual communication links during driving. If vehicle-cloud collaboration is possible and the links are unobstructed, it enters advanced mode; otherwise, it remains in intermediate mode.

[0022] S24. The basic mode achieves basic autonomous driving only through the main autonomous driving system, which directly senses information from the vehicle and outputs control signals. The intermediate mode achieves autonomous driving through the main autonomous driving system and the twin system. The main system generates basic decision results, and the twin system outputs global optimization results to the main system. The system switches through a risk mechanism. The advanced mode introduces vehicle-cloud collaboration on the basis of the intermediate mode, which further guides the twin system to generate advanced global optimization results.

[0023] To further explain, step S3 specifically involves:

[0024] S31. In the primary mode, the vehicle terminal only achieves basic autonomous driving, acquires environmental perception information through on-board sensors, including visual data collected by cameras and point cloud data collected by lidar sensors, and performs high-precision vehicle positioning and navigation through low-orbit satellites.

[0025] S32. The main system uses a visual encoder and a point cloud encoder to process the environmental perception information to obtain visual BEV features and point cloud BEV features, and then uses a multimodal feature fusion algorithm to fuse the visual BEV features and point cloud BEV features to obtain fused features.

[0026] S33. The main system performs basic target detection based on fusion features to obtain the category and location of surrounding targets, and then calculates the distance and cluster density of surrounding targets.

[0027] S34. The main system selects the driving route based on low-orbit satellite positioning and navigation, assesses the risk range of the vehicle based on the distance and density of surrounding targets, and then makes a decision on the path planning and outputs waypoints.

[0028] S35. The main system calculates the speed and direction required to reach the waypoint based on the vehicle's kinematics and dynamics models, and then converts these into lateral and longitudinal control signals.

[0029] To further explain, step S4 specifically involves:

[0030] S41. In intermediate mode, the vehicle terminal uses the main system and the twin system to achieve dual-system autonomous driving. The main system undertakes basic perception, decision-making and control tasks, acquires environmental perception information through on-board sensors, performs high-precision vehicle positioning and navigation through low-orbit satellites, and transmits the data to the twin system. The twin system uses an end-to-end network to output global optimization results. The two systems switch through a risk mechanism.

[0031] S42. Under the risk mechanism, the main system processes and fuses the environmental perception information to obtain fused features. Based on this, it calculates the distance and aggregation density of surrounding targets, further assesses the risk range of the vehicle, generates a risk index by combining the vehicle's pose and speed, and determines the timing of the dual-system switching based on the risk index.

[0032] S43. If the risk index is below the threshold, the twin system is activated. The twin system adopts a modular end-to-end network and periodically obtains dynamically updated vectorized maps from the cloud service center through dual communication links. These maps include traffic signs and lane lines in the field environment where the vehicle terminal is located, thereby enabling more accurate decision-making and planning, and supporting lane-level decision-making and planning at complex intersections.

[0033] S44. The twin system uses the fusion features of the main system as input for object detection and tracking. Then, based on the high-precision positioning of the vehicle provided by low-orbit satellites, it calculates the projection matrix from its own coordinate system to the global coordinate system and projects the detected objects to the correct positions on the map. This predicts and generates the future risk range, thereby performing decision-making path planning and outputting waypoints to the main system. The main system calculates the required speed and direction based on the vehicle's kinematics and dynamics models, and finally converts them into control signals.

[0034] S45. If the risk index is higher than the threshold, it indicates that the current road conditions are relatively complex. At this time, the vehicle speed, field map and historical experience are further combined to judge the critical decision interval of risk. If the interval is greater than the decision delay of the twin system, the twin system continues to perform autonomous driving. Otherwise, it indicates that the decision delay of the twin system will increase the driving risk under the current road conditions. At this time, the system is switched back to the main system for autonomous driving. It directly performs decision path planning based on the risk range, outputs waypoints, and calculates the required speed and direction based on the vehicle kinematic model and dynamic model to generate control signals.

[0035] S46. In intermediate mode, regardless of whether the twin system intervenes, the main system of the vehicle terminal can independently achieve basic autonomous driving, and its structure is relatively simple and can respond quickly in real time. In comparison, the twin system can further achieve end-to-end optimization and output better decision results, but it has a certain computational latency and its real-time performance is lower than that of basic autonomous driving. Therefore, it mainly intervenes in the main system when the risk is low.

[0036] To further explain, step S5 specifically involves:

[0037] S51. In advanced mode, the vehicle terminal uses the main system and twin system to achieve dual-system autonomous driving, and further introduces vehicle-cloud collaboration. The cloud can obtain more general global traffic information and the driving needs of each vehicle terminal, so as to carry out global path advance planning and provide collaborative decision guidance for the vehicle terminal to achieve high-level autonomous driving.

[0038] S52. In addition to completing various tasks in the intermediate mode, the main system further transmits environmental perception data and high-precision vehicle positioning data to the cloud service center through the main link.

[0039] S53, the cloud service center clusters multiple vehicle terminals into groups. Within each group, it preprocesses the latest data of each vehicle terminal. First, it extracts the point cloud data of each vehicle terminal and maps it to the global coordinate system based on the vehicle's positioning data to achieve preliminary point cloud fusion. Then, it generates field point cloud BEV features through a point cloud encoder. After that, it extracts the visual data of each vehicle terminal and generates field visual BEV features based on vehicle positioning and visual encoder. Finally, it uses a multimodal feature fusion algorithm to fuse the field point cloud BEV features and the field visual BEV features to obtain field fusion features, while dynamically updating the field map.

[0040] S54, the cloud service center performs field target detection and tracking based on field fusion characteristics, and calibrates the field target position through low-orbit satellites to predict the target trajectory and generate the future risk range of the field;

[0041] S55: The cloud service center makes high-dimensional optimal decisions based on the driving needs of each vehicle terminal in the group, the future risk range of the field, and the traffic flow optimization objectives, and realizes global path advance planning. Then, the advance path planning results and the future risk range of the field are transmitted back to each vehicle terminal through the main link.

[0042] S56. The vehicle terminal receives the advanced path planning results and the future risk range of the field from the cloud service center through the twin system. Based on this, it corrects its own decision results, achieves better decision path planning, and outputs waypoints to the main system. Based on the vehicle kinematics model and dynamics model, it calculates speed and direction, and finally outputs control signals.

[0043] S57. If the vehicle terminal suddenly enters a high-risk road section, it switches to the main system to drive with full authority, directly making decision-making path planning based on the risk range, outputting waypoints, calculating speed and direction, and generating control signals. At the same time, the main system continues to transmit data to the cloud service center to achieve vehicle-cloud collaborative high-level autonomous driving again when leaving the high-risk road section. If the vehicle terminal stays in the high-risk road section for a long time and exceeds the time waiting threshold, the main system stops transmitting data to the cloud service center to release communication bandwidth and reduce communication pressure in the field.

[0044] To further explain, step S6 specifically involves:

[0045] S61. In all driving modes, the vehicle terminal uses the main system as the underlying architecture for autonomous driving. In abnormal conditions, such as system crashes or system upgrades, the vehicle terminal should still have driving capabilities and be able to provide takeover options when the system recovers.

[0046] S62. In each driving mode, the controller is independent of the main system. It can accept bidirectional signal input from the main system and manual operation, and monitor the function of the main system through a supervision mechanism.

[0047] S63. In the supervision mechanism, perception test data is periodically input to the main system, and control test signals output by it are received. If the control test signal is issued normally and within the normal range, it indicates that the main system is functioning normally and can continue to execute the autonomous driving task. Otherwise, it indicates that the main system is in an abnormal working condition and it is difficult to continue the autonomous driving task. At this time, the controller detects the corresponding situation, reminds the user to directly intervene in driving through an alarm sound, and uses manual operation signals to override permissions.

[0048] S64. After the user directly intervenes in driving, he can directly output manual operation signals to the controller through components such as accelerator, brake, and steering wheel, thereby continuing to control the vehicle terminal.

[0049] S65. After the main system resumes functioning, it restarts sending control test signals to the controller. The controller then begins to receive bidirectional signals from the main system and manual operation, and provides the user with an automatic driving switching option through the vehicle system. If the user switches back to automatic driving mode, the main system takes over the driving state; otherwise, it continues to maintain the user's operating state, thereby achieving safer driving.

[0050] This invention also proposes an integrated space-ground autonomous driving system for intelligent vehicles, including a vehicle terminal, a cloud service center, and dual communication links; the vehicle terminal is able to transmit information to the cloud service center through the dual communication links.

[0051] The vehicle terminal includes an autonomous driving system and a manual driving system. The autonomous driving system and the manual driving system are switched under the supervision of a controller. When the controller detects that the autonomous driving system is normal, the autonomous driving system controls the vehicle. When the controller detects that the autonomous driving system is abnormal, the manual driving system controls the vehicle.

[0052] The autonomous driving system includes a main system and a twin system. The main system and the twin system dynamically switch between three autonomous driving modes based on the vehicle's status: a basic mode, an intermediate mode, and an advanced mode. In the basic mode, only the main system achieves basic autonomous driving, directly sensing information from the vehicle and outputting control signals. In the intermediate mode, the main system and the twin system cooperate to achieve autonomous driving. The main system generates basic decision results, and the twin system outputs global optimization results to the main system. A risk mechanism enables system switching. In the advanced mode, based on the intermediate mode, vehicle-cloud collaboration is implemented. The cloud service center further guides the twin system to generate advanced global optimization results and transmits them to the main system, which then outputs control signals.

[0053] The manual driving system allows the driver to directly control the accelerator, brakes, and steering wheel;

[0054] The dual communication links include a mobile communication network and a satellite communication network. The vehicle terminal and the cloud service center periodically send data packets to determine the reachability and latency of the dual communication links. The link with the lowest latency is selected as the primary link for data transmission, while the other link is a backup link. When the primary link is unreachable or has excessive latency, the backup link and the primary link are interchanged.

[0055] Furthermore, the main system and the twin system dynamically switch the autonomous driving mode to primary mode, intermediate mode and advanced mode according to the vehicle status. This switching process adopts the content of S2 to S5 above.

[0056] Furthermore, the switching between the autonomous driving system and the manual driving system is supervised by a controller, and this supervision mechanism is implemented according to the content of S6 above.

[0057] The beneficial effects of this invention are:

[0058] 1. This invention accesses the cloud service center via dual links of mobile communication and satellite communication, ensuring communication redundancy and stability. In the event that the main link becomes unreachable or has excessive latency, the backup link can take over in a timely manner, ensuring the continuous operation of the external communication system.

[0059] 2. This invention designs three autonomous driving modes: primary, intermediate, and advanced. These modes can be dynamically switched according to the real-time vehicle status and road conditions, ensuring that the optimal driving strategy can be provided in driving environments of varying complexity, thereby improving the system's adaptability and flexibility.

[0060] 3. Through the design of a risk mechanism, this invention can prioritize the main system for rapid decision-making in complex road conditions with high real-time requirements, while introducing a twin system for in-depth optimization in low-risk situations. This effectively balances the relationship between real-time response and decision optimization, thereby improving the overall system performance.

[0061] 4. This invention utilizes multimodal sensor fusion and high-precision positioning technology, combined with global data processing in the cloud, to achieve accurate target detection, trajectory prediction, and risk assessment, making the autonomous driving system more intelligent and forward-looking in path planning, and significantly improving driving safety and efficiency.

[0062] 5. This invention ensures that when the main system fails abnormally, the controller can promptly monitor and prompt the user to intervene in driving, ensuring that the vehicle has safe driving capabilities under any circumstances, thereby improving the system's fault tolerance, safety, and reliability. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the overall method flow of the present invention;

[0064] Figure 2 This is a schematic diagram illustrating the specific operation process of the autonomous driving system of the present invention;

[0065] Figure 3 This is a schematic diagram of the safe driving system of the present invention; Detailed Implementation

[0066] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0067] like Figure 1 As shown, the intelligent vehicle autonomous driving method of the present invention, which integrates space, ground, and autonomous driving, includes the following steps:

[0068] S1. The vehicle terminal starts up, calibrates the vehicle positioning information, and connects to the cloud service center through dual communication links;

[0069] S2. Activate the main autonomous driving system and twin system, and dynamically switch autonomous driving modes according to the vehicle status, including primary mode, intermediate mode and advanced mode.

[0070] S3. If the autonomous driving mode is the primary mode, then the basic autonomous driving strategy and method under the primary mode will be activated.

[0071] S4. If the autonomous driving mode is intermediate mode, then activate the dual-system autonomous driving strategy and method in intermediate mode.

[0072] S5. If the autonomous driving mode is advanced mode, then activate the vehicle-cloud collaborative autonomous driving strategy and method in advanced mode.

[0073] S6. If the main system fails abnormally, activate the emergency driving method.

[0074] To further explain, step S1 specifically includes:

[0075] The vehicle terminal starts up, calibrates the vehicle positioning information, and connects to the cloud service center via dual communication links.

[0076] S11. The vehicle terminal starts, the on-board components perform self-test, the on-board sensors are initialized and calibrated;

[0077] S12. The vehicle terminal opens the network interface, detects and connects to the mobile communication network and the satellite communication network, sends a data packet for the first time to determine the reachability of the dual communication links, connects to the cloud service center through the dual links, and calibrates the vehicle positioning information at the same time.

[0078] S13. The vehicle terminal and cloud service center periodically send data packets to determine the reachability and latency of the two communication links. The link with the lowest latency is selected as the primary link for data transmission, while the other link is the communication backup link. When the primary link is unreachable or the latency is too high, the roles of the communication backup link and the primary link are swapped.

[0079] Combination Figure 2 As shown, the specific process of the operation of the autonomous driving system of the present invention will be described below.

[0080] To further explain, step S2 specifically involves:

[0081] The main autonomous driving system and twin system are activated, and the autonomous driving mode is dynamically switched according to the vehicle status.

[0082] S21. Read the vehicle's autonomous driving configuration file, start the main autonomous driving system, run the main system's self-test data to ensure the availability and accuracy of basic autonomous driving, and enter the primary mode;

[0083] S22. After ensuring the stable operation of the main autonomous driving system, start the twin system, load the end-to-end network, and run the twin system self-test data. If the self-test passes, enter the intermediate mode and achieve autonomous driving through the dual systems. Otherwise, stay in the primary mode and rely solely on the main system to achieve autonomous driving.

[0084] S23. When the autonomous driving system is in intermediate mode, it attempts to obtain vehicle-cloud collaborative information through dual communication links during driving. If vehicle-cloud collaboration is possible and the links are unobstructed, it enters advanced mode; otherwise, it remains in intermediate mode.

[0085] S24. The basic mode achieves basic autonomous driving only through the main autonomous driving system, which directly outputs control signals from the vehicle's perception information. The intermediate mode achieves autonomous driving through the joint operation of the main autonomous driving system and the twin system. The main system generates basic decision results, and the twin system outputs global optimization results to the main system. The system switches through a risk mechanism. The advanced mode introduces vehicle-cloud collaboration on the basis of the intermediate mode, which further guides the twin system to generate advanced global optimization results.

[0086] To further explain, step S3 specifically involves:

[0087] Basic autonomous driving strategies and methods in the initial mode;

[0088] S31. In the primary mode, the vehicle terminal only achieves basic autonomous driving, acquires environmental perception information through on-board sensors, including visual data collected by cameras and point cloud data collected by lidar sensors, and performs high-precision vehicle positioning and navigation through low-orbit satellites.

[0089] S32. The main system uses a visual encoder and a point cloud encoder to process the environmental perception information to obtain visual BEV features and point cloud BEV features, and then uses a multimodal feature fusion algorithm to fuse the visual BEV features and point cloud BEV features to obtain fused features.

[0090] S33. The main system performs basic target detection based on fusion features to obtain the category and location of surrounding targets, and then calculates the distance and cluster density of surrounding targets.

[0091] S34. The main system selects the driving route based on low-orbit satellite positioning and navigation, assesses the risk range of the vehicle based on the distance and density of surrounding targets, and then makes a decision on the path planning and outputs waypoints.

[0092] S35. The main system calculates the speed and direction required to reach the waypoint based on the vehicle's kinematics and dynamics models, and then converts these into lateral and longitudinal control signals.

[0093] To further explain, step S4 specifically involves:

[0094] Dual-system autonomous driving strategies and methods in intermediate mode;

[0095] S41. In intermediate mode, the vehicle terminal uses the main system and the twin system to achieve dual-system autonomous driving. The main system undertakes basic perception, decision-making and control tasks, acquires environmental perception information through on-board sensors, performs high-precision vehicle positioning and navigation through low-orbit satellites, and transmits the data to the twin system. The twin system uses an end-to-end network to output global optimization results. The two systems switch through a risk mechanism.

[0096] S42. Under the risk mechanism, the main system processes and fuses the environmental perception information to obtain fused features. Based on this, it calculates the distance and aggregation density of surrounding targets, further assesses the vehicle risk range, generates a risk index by combining the vehicle pose and speed, and determines the timing of the dual-system switching based on the index.

[0097] S43. If the risk index is below the threshold, the twin system is activated. The twin system adopts a modular end-to-end network and periodically obtains dynamically updated vectorized maps from the cloud service center through dual communication links. These maps include traffic signs and lane lines in the field environment where the vehicle terminal is located, thereby enabling more accurate decision-making and planning, and supporting lane-level decision-making and planning at complex intersections.

[0098] S44. The twin system uses the fusion features of the main system as input for object detection and tracking. Then, based on the high-precision positioning of the vehicle provided by low-orbit satellites, it calculates the projection matrix from its own coordinate system to the global coordinate system and projects the detected objects to the correct positions on the map. This predicts and generates the future risk range, thereby performing decision-making path planning and outputting waypoints to the main system. The main system calculates the required speed and direction based on the vehicle's kinematics and dynamics models, and finally converts them into control signals.

[0099] S45. If the risk index is higher than the threshold, it indicates that the current road conditions are relatively complex. At this time, the vehicle speed, field map and historical experience are further combined to judge the critical decision interval of risk. If the interval is greater than the decision delay of the twin system, the twin system continues to perform autonomous driving. Otherwise, it indicates that the decision delay of the twin system will increase the driving risk under the current road conditions. At this time, the system is switched back to the main system for autonomous driving. It directly performs decision path planning based on the risk range, outputs waypoints, and calculates the required speed and direction based on the vehicle kinematic model and dynamic model to generate control signals.

[0100] S46. In intermediate mode, regardless of whether the twin system intervenes, the main system of the vehicle terminal can independently achieve basic autonomous driving, and its structure is relatively simple and can respond quickly in real time. In comparison, the twin system can further achieve end-to-end optimization and output better decision results, but it has a certain computational latency and its real-time performance is lower than that of basic autonomous driving. Therefore, it mainly intervenes in the main system when the risk is low.

[0101] To further explain, step S5 specifically involves:

[0102] Advanced mode of vehicle-cloud collaborative autonomous driving strategies and methods;

[0103] S51. In advanced mode, the vehicle terminal uses the main system and twin system to achieve dual-system autonomous driving, and further introduces vehicle-cloud collaboration. The cloud can obtain more general global traffic information and the driving needs of each vehicle terminal, so as to carry out global path advance planning and provide collaborative decision guidance for the vehicle terminal to achieve high-level autonomous driving.

[0104] S52. In addition to completing various tasks in the intermediate mode, the main system further transmits environmental perception data and high-precision vehicle positioning data to the cloud service center through the main link.

[0105] S53, the cloud service center clusters multiple vehicle terminals into groups. Within each group, it preprocesses the latest data of each vehicle terminal. First, it extracts the point cloud data of each vehicle terminal and maps it to the global coordinate system based on the vehicle's positioning data to achieve preliminary point cloud fusion. Then, it generates field point cloud BEV features through a point cloud encoder. After that, it extracts the visual data of each vehicle terminal and generates field visual BEV features based on vehicle positioning and visual encoder. Finally, it uses a multimodal feature fusion algorithm to fuse the field point cloud BEV features and the field visual BEV features to obtain field fusion features, while dynamically updating the field map.

[0106] S54, the cloud service center performs field target detection and tracking based on field fusion characteristics, and calibrates the field target position through low-orbit satellites to predict the target trajectory and generate the future risk range of the field;

[0107] S55: The cloud service center makes high-dimensional optimal decisions based on the driving needs of each vehicle terminal in the group, the future risk range of the field, and the traffic flow optimization objectives, and realizes global path advance planning. Then, the advance path planning results and the future risk range of the field are transmitted back to each vehicle terminal through the main link.

[0108] S56. The vehicle terminal receives the advanced path planning results and the future risk range of the field from the cloud service center through the twin system. Based on this, it corrects its own decision results, achieves better decision path planning, and outputs waypoints to the main system. Based on the vehicle kinematics model and dynamics model, it calculates speed and direction, and finally outputs control signals.

[0109] S57. If the vehicle terminal suddenly enters a high-risk road section, it switches to the main system to drive with full authority, directly making decision-making and path planning based on the risk range, outputting waypoints, calculating speed and direction, and generating control signals. At the same time, the main system continues to transmit data to the cloud service center to achieve vehicle-cloud collaborative high-level autonomous driving again when leaving the high-risk road section. If the vehicle terminal stays in the high-risk road section for a long time and exceeds the time waiting threshold, the main system stops transmitting data to the cloud service center to release communication bandwidth and reduce communication pressure in the field.

[0110] To further explain, step S6 specifically involves:

[0111] like Figure 3 As shown, this describes the driving method when the main system fails abnormally;

[0112] S61. In all driving modes, the vehicle terminal uses the main system as the underlying architecture for autonomous driving. In abnormal conditions, such as system crashes or system upgrades, the vehicle terminal should still have driving capabilities and be able to provide takeover options when the system recovers.

[0113] S62. In each driving mode, the controller is independent of the main system. It can accept bidirectional control signal input from the main system and manual operation, and monitor the function of the main system through a monitoring mechanism.

[0114] S63. In the supervision mechanism, the controller periodically inputs perception test data to the main system and receives the control test signal output by it. If the control test signal is issued normally and within the normal range, it indicates that the main system is functioning normally and can continue to execute the autonomous driving task. Otherwise, it indicates that the main system is in an abnormal working condition and it is difficult to continue the autonomous driving task. At this time, the controller detects the corresponding situation, reminds the user to directly intervene in driving through an alarm sound, and uses manual operation signals to override the permissions.

[0115] S64. After the user directly intervenes in driving, he can directly output manual operation signals to the controller through components such as accelerator, brake, and steering wheel, thereby continuing to control the vehicle terminal.

[0116] S65. After the main system resumes functioning, it restarts sending control test signals to the controller. The controller then begins to receive bidirectional signals from the main system and manual operation, and provides the user with an automatic driving switching option through the vehicle system. If the user switches back to automatic driving mode, the main system takes over the driving state; otherwise, it continues to maintain the user's operating state, thereby achieving safer driving.

[0117] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent methods or modifications that do not depart from the technology of the present invention should be included within the scope of protection of the present invention.

Claims

1. A space-ground integrated intelligent vehicle autonomous driving method, characterized in that, include: S1. The vehicle terminal starts up, calibrates the vehicle positioning information, and connects to the cloud service center through dual communication links; S2. Activate the main autonomous driving system and twin system, and dynamically switch autonomous driving modes according to the vehicle status; S3. If the autonomous driving mode is the primary mode, then the basic autonomous driving strategy and method under the primary mode will be activated. S4. If the autonomous driving mode is intermediate mode, then activate the dual-system autonomous driving strategy and method in intermediate mode. S5. If the autonomous driving mode is advanced mode, then activate the vehicle-cloud collaborative autonomous driving strategy and method in advanced mode. S6. If the main autonomous driving system fails abnormally, activate the emergency driving method; Step S2 specifically involves: S21. Read the vehicle's autonomous driving configuration file, start the main autonomous driving system, run the main system's self-test data to ensure the availability and accuracy of basic autonomous driving, and enter the primary mode; S22. After ensuring the stable operation of the main autonomous driving system, start the twin system, load the end-to-end network, and run the twin system self-test data. If the self-test passes, enter the intermediate mode and achieve autonomous driving through the dual systems. Otherwise, stay in the primary mode and rely solely on the main system to achieve autonomous driving. S23. When the autonomous driving system is in intermediate mode, it attempts to obtain vehicle-cloud collaborative information through dual communication links during driving. If vehicle-cloud collaboration is possible and the links are unobstructed, it enters advanced mode; otherwise, it remains in intermediate mode. S24. In the primary mode, basic autonomous driving is achieved solely through the main autonomous driving system, which directly senses information from the vehicle and outputs control signals. In the intermediate mode, autonomous driving is achieved jointly through the main autonomous driving system and the twin system. The main system generates basic decision results, and the twin system outputs global optimization results to the main system. System switching is achieved through a risk mechanism. In the advanced mode, vehicle-cloud collaboration is introduced on the basis of the intermediate mode to further guide the twin system in generating advanced global optimization results. The specific implementation of step S3 includes: S31. In the primary mode, the vehicle terminal only achieves basic autonomous driving, acquires environmental perception information through on-board sensors, including visual data collected by cameras and point cloud data collected by lidar sensors, and performs high-precision vehicle positioning and navigation through low-orbit satellites. S32. The main system uses a visual encoder and a point cloud encoder to process the environmental perception information to obtain visual BEV features and point cloud BEV features, and then uses a multimodal feature fusion algorithm to fuse the visual BEV features and point cloud BEV features to obtain fused features. S33. The main system performs basic target detection based on fusion features to obtain the category and location of surrounding targets, and then calculates the distance and cluster density of surrounding targets. S34. The main system selects the driving route based on low-orbit satellite positioning and navigation, assesses the risk range of the vehicle based on the distance and density of surrounding targets, and then makes a decision on the path planning and outputs waypoints. S35. The main system calculates the speed and direction required to reach the waypoint based on the vehicle's kinematics and dynamics models, and then converts these into lateral and longitudinal control signals. The specific implementation of step S4 includes: S41. In intermediate mode, the vehicle terminal uses the main system and the twin system to achieve dual-system autonomous driving. The main system undertakes basic perception, decision-making and control tasks, acquires environmental perception information through on-board sensors, performs high-precision vehicle positioning and navigation through low-orbit satellites, and transmits the data to the twin system. The twin system uses an end-to-end network to output global optimization results. The two systems switch through a risk mechanism. S42. Under the risk mechanism, the main system processes and fuses the environmental perception information to obtain fused features. Based on this, it calculates the distance and aggregation density of surrounding targets, further assesses the risk range of the vehicle, generates a risk index by combining the vehicle's pose and speed, and determines the timing of the dual-system switching based on the risk index. S43. If the risk index is below the threshold, the twin system is activated. The twin system adopts a modular end-to-end network and periodically obtains dynamically updated vectorized maps from the cloud service center through dual communication links. These maps include traffic signs and lane lines in the field environment where the vehicle terminal is located, thereby enabling more accurate decision-making and planning, and supporting lane-level decision-making and planning at complex intersections. S44. The twin system uses the fusion features of the main system as input for object detection and tracking. Then, based on the high-precision positioning of the vehicle provided by low-orbit satellites, it calculates the projection matrix from its own coordinate system to the global coordinate system and projects the detected objects to the correct positions on the map. This predicts and generates the future risk range, thereby performing decision-making path planning and outputting waypoints to the main system. The main system calculates the required speed and direction based on the vehicle's kinematics and dynamics models, and finally converts them into control signals. S45. If the risk index is higher than the threshold, it indicates that the current road conditions are relatively complex. At this time, the vehicle speed, field map and historical experience are further combined to judge the critical decision interval of risk. If the interval is greater than the decision delay of the twin system, the twin system continues to perform autonomous driving. Otherwise, it indicates that the decision delay of the twin system will increase the driving risk under the current road conditions. At this time, the system is switched back to the main system for autonomous driving. It directly performs decision path planning based on the risk range, outputs waypoints, and calculates the required speed and direction based on the vehicle kinematic model and dynamic model to generate control signals. The specific implementation of step S5 includes: S51. In advanced mode, the vehicle terminal uses the main system and twin system to achieve dual-system autonomous driving, and further introduces vehicle-cloud collaboration. It uses the cloud service center to obtain more generalized global traffic information and the driving needs of each vehicle terminal, so as to carry out global path advance planning and provide collaborative decision guidance for the vehicle terminal to achieve high-level autonomous driving. S52. In addition to completing various tasks in the intermediate mode, the main system further transmits environmental perception data and high-precision vehicle positioning data to the cloud service center through the main link. S53, the cloud service center clusters multiple vehicle terminals into groups. Within each group, it preprocesses the latest data of each vehicle terminal. First, it extracts the point cloud data of each vehicle terminal and maps it to the global coordinate system based on the vehicle's positioning data to achieve preliminary point cloud fusion. Then, it generates field point cloud BEV features through a point cloud encoder. After that, it extracts the visual data of each vehicle terminal and generates field visual BEV features based on vehicle positioning and visual encoder. Finally, it uses a multimodal feature fusion algorithm to fuse the field point cloud BEV features and the field visual BEV features to obtain field fusion features, while dynamically updating the field map. S54, the cloud service center performs field target detection and tracking based on field fusion characteristics, and calibrates the field target position through low-orbit satellites to predict the target trajectory and generate the future risk range of the field; S55: The cloud service center makes high-dimensional optimal decisions based on the driving needs of each vehicle terminal in the group, the future risk range of the field, and the traffic flow optimization objectives, and realizes global path advance planning. Then, the advance path planning results and the future risk range of the field are transmitted back to each vehicle terminal through the main link. S56. The vehicle terminal receives the advanced path planning results and the future risk range of the field from the cloud service center through the twin system. Based on this, it corrects its own decision results, achieves better decision path planning, and outputs waypoints to the main system. The main system calculates speed and direction based on the vehicle kinematics model and dynamics model, and finally outputs control signals. S57. If the vehicle terminal suddenly enters a high-risk road section, it switches to the main system to drive with full authority, directly making decision-making and path planning based on the risk range, outputting waypoints, calculating speed and direction, and generating control signals. At the same time, the main system continues to transmit data to the cloud service center to achieve vehicle-cloud collaborative high-level autonomous driving again when leaving the high-risk road section. If the vehicle terminal stays in the high-risk road section for a long time and exceeds the time waiting threshold, the main system stops transmitting data to the cloud service center to release communication bandwidth and reduce communication pressure in the field. The specific implementation of step S6 includes: S61. In all driving modes, the vehicle terminal uses the main system as the underlying architecture for autonomous driving. In abnormal conditions, including system crashes and system upgrades, the vehicle terminal should still have driving capabilities and be able to provide takeover options when the system recovers. S62. In each driving mode, the controller is independent of the main system. It can accept bidirectional control signal input from the main system and manual operation, and monitor the function of the main system through a monitoring mechanism. S63. In the supervision mechanism, perception test data is periodically input to the main system, and control test signals output by it are received. If the control test signal is issued normally and within the normal range, it indicates that the main system is functioning normally and can continue to execute the autonomous driving task. Otherwise, it indicates that the main system is in an abnormal working condition and it is difficult to continue the autonomous driving task. At this time, the controller detects the corresponding situation, reminds the user to directly intervene in driving through an alarm sound, and uses manual operation signals to override permissions. S64. After the user directly intervenes in driving, he / she can directly output manual operation signals to the controller through the accelerator, brake and steering wheel, thereby continuing to control the car terminal. S65. After the main system resumes functioning, it restarts sending control test signals to the controller. The controller then begins to receive bidirectional signals from the main system and manual operation, and provides the user with an automatic driving switching option through the vehicle system. If the user switches back to automatic driving mode, the main system takes over the driving state; otherwise, it continues to maintain the user's operating state, thereby achieving safer driving.

2. The intelligent vehicle autonomous driving method with integrated space-ground coordination according to claim 1, characterized in that, Step S1 specifically includes: S11. The vehicle terminal starts, the on-board components perform self-test, the on-board sensors are initialized and calibrated; S12. The vehicle terminal opens the network interface, detects and connects to the dual link composed of the mobile communication network and the satellite communication network, sends a data packet for the first time to determine the reachability of the dual communication link, connects to the cloud service center through the dual link, and calibrates the vehicle positioning information at the same time. S13. The vehicle terminal and cloud service center periodically send data packets to determine the reachability and latency of the two communication links. The link with the lowest latency is selected as the primary link for data transmission, while the other link is the communication backup link. When the primary link is unreachable or the latency is too high, the roles of the communication backup link and the primary link are swapped.

3. A space-ground integrated collaborative intelligent vehicle autonomous driving system that implements the space-ground integrated collaborative intelligent vehicle autonomous driving method according to any one of claims 1-2, characterized in that, It includes a vehicle terminal, a cloud service center, and dual communication links; the vehicle terminal is able to transmit information with the cloud service center through the dual communication links. The vehicle terminal includes an autonomous driving system and a manual driving system. The autonomous driving system and the manual driving system are switched under the supervision of a controller. When the controller detects that the autonomous driving system is normal, the autonomous driving system controls the vehicle. When the controller detects that the autonomous driving system is abnormal, the manual driving system controls the vehicle. The autonomous driving system includes a main system and a twin system. The main system and the twin system dynamically switch between three autonomous driving modes based on the vehicle's status: a basic mode, an intermediate mode, and an advanced mode. In the basic mode, only the main system achieves basic autonomous driving, directly sensing information from the vehicle and outputting control signals. In the intermediate mode, the main system and the twin system cooperate to achieve autonomous driving. The main system generates basic decision results, and the twin system outputs global optimization results to the main system. A risk mechanism enables system switching. In the advanced mode, based on the intermediate mode, vehicle-cloud collaboration is implemented. The cloud service center further guides the twin system to generate advanced global optimization results and transmits them to the main system, which then outputs control signals. The manual driving system allows the driver to directly control the accelerator, brake, and steering wheel; The dual communication links include a mobile communication network and a satellite communication network. The vehicle terminal and the cloud service center periodically send data packets to determine the reachability and latency of the dual communication links. The link with the lowest latency is selected as the primary link for data transmission, while the other link is a backup link. When the primary link is unreachable or has excessive latency, the backup link and the primary link are interchanged.

4. The integrated space-ground autonomous driving system for intelligent vehicles according to claim 3, characterized in that, The main system and the twin system dynamically switch the autonomous driving mode to primary mode, intermediate mode and advanced mode according to the vehicle status. The switching process adopts the content of S2 to S5 as described in any one of claims 1-2.

5. The integrated space-ground autonomous driving system for intelligent vehicles according to claim 3, characterized in that, The automatic driving system and the manual driving system are switched under the supervision of a controller, and the supervision mechanism is implemented in accordance with the content of S6 as described in any one of claims 1-2.

Images