Intelligent driving closed-loop control method and device based on pre-training large model and medium

Abstract

The invention provides an intelligent driving closed-loop control method based on a pre-training large model. The intelligent driving closed-loop control method comprises the following steps: S1, obtaining full-quantity heterogeneous input data in a vehicle driving process; s2, performing feature coding on the total heterogeneous input data to generate feature data suitable for pre-training large model processing; s3, inputting the encoded feature data into a pre-trained large model for feature extraction, and outputting a key value pair; s4, constructing a plurality of task head sharing key value pairs in a cascade form, and generating intention-level and track-level outputs; s5, outputting and generating a vehicle control instruction according to the intention level and the track level to control the driving state of the vehicle; according to the method, full-quantity heterogeneous input features in a closed-loop form are constructed, so that the use efficiency of data is remarkably improved, and the most comprehensive feature representation is extracted from multi-source input, so that an output result with higher quality is generated; according to the method, the problems of information loss and accumulative errors in a traditional modular system are effectively solved, and the accuracy and reliability of decision planning are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to an intelligent driving closed-loop control method, device and medium based on a pre-trained large model. Background Art

[0002] In the current intelligent driving field, traditional modular intelligent driving solutions include modules such as perception prediction, decision planning, and control execution. Although these modules are complementary in function, their independence may lead to information loss and cumulative errors when information is transmitted between modules, and the accuracy and reliability of decision planning are low. The perception module is responsible for identifying dynamic and static obstacles and passing them to the prediction module. The prediction module predicts the environment based on the perception information and provides decision references for the planning module. The decision planning module then converts the prediction information into specific trajectory planning results, and the tracking control module tracks the trajectory planning results to generate vehicle control instructions. Although this segmented processing method has advantages in processing specific tasks, it often cannot ensure global optimality as a whole, and its adaptability and flexibility are limited when facing complex dynamic environments.

[0003] Although the fully end-to-end intelligent driving solution simplifies the system architecture in theory and can process directly from input to output, it has some significant disadvantages in practical applications, such as low interpretability and debuggability, huge data requirements and poor generalization, and the number of model parameters also limits the real-time deployment of the algorithm. In addition, the autonomous driving system must comply with strict regulatory standards. Due to its opaque decision-making process, the end-to-end system may find it difficult to meet these regulatory requirements, especially when it is necessary to prove that the system decision logic meets safety requirements. Summary of the invention

[0004] The present invention proposes an intelligent driving closed-loop control method, device and medium based on a pre-trained large model, which solves the problems of low accuracy and reliability of decision-making planning in traditional modular intelligent driving solutions due to information loss and cumulative errors.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A first aspect of the present invention provides an intelligent driving closed-loop control method based on a pre-trained large model, comprising the following steps:

[0007] S1, obtaining full heterogeneous input data during vehicle driving, wherein the full heterogeneous input data includes closed-loop input data, environment perception results of previous N frames, local mapping results of the current frame, and navigation information of the current frame;

[0008] S2, feature encodes all heterogeneous input data to generate feature data suitable for processing by the pre-trained large model;

[0009] S3, input the encoded feature data into the pre-trained large model with frozen parameters for feature extraction, and output key-value pairs;

[0010] S4, builds multiple task heads in cascade form to share the key-value pairs of the large model output. Each task head holds an independent trainable query network, which is used to generate intent-level and trajectory-level outputs respectively;

[0011] S5, generates vehicle control instructions based on the intention-level output and trajectory-level output to control the vehicle's driving state, and feeds back the trajectory-level output, the vehicle's current control state and posture information as closed-loop input data to the next frame of input data of the large model, forming a closed-loop feedback.

[0012] Specifically, in step S2, after the input data from different sources are independently feature encoded and information compressed, an ensemble learning algorithm is used to combine them into full input features.

[0013] Specifically, in step S4, the multiple task heads in cascade form include two layers of task heads, the first layer of task heads includes an other vehicle lateral intention prediction task head, an other vehicle longitudinal intention prediction task head and a scene recognition task head, and the second layer of task heads is a self-vehicle trajectory prediction task head; the outputs of the other vehicle lateral intention prediction task head, the other vehicle longitudinal intention prediction task head and the scene recognition task head and the key-value pairs of the large model output are used as the input of the self-vehicle trajectory prediction task head, and the self-vehicle trajectory prediction task head is used to output the self-vehicle intention prediction result and the self-vehicle trajectory planning result.

[0014] Furthermore, the self-vehicle trajectory planning result output by the self-vehicle trajectory prediction task head is verified by the safety decision posterior network. If the safety condition is met, the self-vehicle trajectory planning result and the self-vehicle intention prediction result are sent to the vehicle control module for generating vehicle control instructions; if the safety condition is not met, a backup trajectory planning result is generated, and the backup trajectory planning result and the self-vehicle intention prediction result are sent to the vehicle control module for generating vehicle control instructions.

[0015] Specifically, the safety decision posterior network evaluates whether the vehicle trajectory planning result meets the safety condition by means of threshold judgment.

[0016] Specifically, if the safety condition is not met, the self-vehicle following trajectory in the lane is generated as a fallback trajectory by sampling.

[0017] Specifically, each of the task heads is connected to an Adapter module for fine-tuning the model.

[0018] Specifically, the vehicle control instructions include driving instructions, braking instructions, steering instructions, lighting control instructions, human-computer interaction control instructions, seat adjustment instructions, and suspension adjustment instructions.

[0019] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the intelligent driving closed-loop control method are implemented.

[0020] A third aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the intelligent driving closed-loop control method are implemented.

[0021] Compared with the prior art, the technical effects of the present invention are:

[0022] (1) The present invention integrates the vehicle's historical trajectory planning results, environmental perception results, local mapping results, navigation information, and dynamic information such as vehicle posture by constructing a closed-loop full-scale heterogeneous input feature. This comprehensive information utilization strategy significantly improves the efficiency of data use and ensures that the most comprehensive feature representation is extracted from multi-source inputs, thereby generating higher-quality output results. This method effectively overcomes the problems of information loss and cumulative errors in traditional modular systems and improves the accuracy and reliability of decision-making planning.

[0023] (2) By introducing a cascaded multi-task head and a safety decision posterior network, the present invention can not only provide accurate output at the intent level and trajectory level in the decision-making process, but also ensure that each generated trajectory undergoes a strict safety assessment; only verified trajectories can be used for actual vehicle control, which greatly improves the safety of the system and the reliability of trajectory execution;

[0024] (3) The present invention can continuously optimize and adjust the performance of the intelligent driving large model by using the real-time control parameter feedback of the vehicle as the input of the next frame of the model; this closed-loop structure not only improves the adaptability of the model to environmental changes, but also enables the system to self-correct and optimize during continuous operation, thereby improving the overall driving performance and response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a flow chart of the closed-loop control method of intelligent driving based on a pre-trained large model of the present invention;

[0027] Figure 2A data flow diagram of the intelligent driving closed-loop control method in an embodiment of the present invention;

[0028] Figure 3 This is a data flow diagram of multiple intelligent driving task heads in cascade form in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] Reference Figures 1 to 3 In a first aspect, this embodiment provides an intelligent driving closed-loop control method based on a pre-trained large model, comprising the following steps:

[0031] S1, obtaining full heterogeneous input data during vehicle driving, wherein the full heterogeneous input data includes closed-loop input data, environmental perception results of the previous N frames, local mapping results of the current frame, and navigation information of the current frame; the closed-loop input data includes historical trajectory planning results of the previous N frames, the current control state quantity and posture information of the vehicle;

[0032] S2, inputting the full amount of heterogeneous input data into a pre-encoder for feature encoding to generate feature data suitable for processing by a pre-trained large model; the pre-encoder is a learnable module that is trained and optimized to improve the expression efficiency of features;

[0033] S3: Input the encoded feature data into the pre-trained large model with frozen parameters for full feature extraction. Freezing parameters ensures the stability of the model and the effective convergence of the training process. The output of the model is the Key-Value pair (i.e., key-value pair) used for subsequent task heads, providing consistent benchmark features for subsequent different task heads.

[0034] S4, builds multiple intelligent driving task heads in cascade form, sharing the key-value pairs output by the large model. Each task head holds an independent trainable query network, which is used to generate intent-level and trajectory-level outputs respectively;

[0035] S5, generates vehicle control instructions based on the intention-level output and trajectory-level output to control the actions of vehicle control modules such as steering, drive, brake, light and suspension, and feeds back the trajectory-level output, the vehicle's current control state (i.e., vehicle control instructions) and posture information as closed-loop input data to the next frame of input data of the large model, forming a closed-loop feedback.

[0036] Specifically, in step S2, after the input data from different sources are independently feature encoded and information compressed, an ensemble learning algorithm is used to combine them into full input features.

[0037] Specifically, in step S4, the multiple task heads in the cascade form include two layers of task heads, the first layer of task heads includes the other vehicle lateral intention prediction task head, the other vehicle longitudinal intention prediction task head and the scene recognition task head, and the second layer of task heads is the own vehicle trajectory prediction task head; the outputs of the other vehicle lateral intention prediction task head, the other vehicle longitudinal intention prediction task head and the scene recognition task head and the key-value pairs of the large model output are used as the inputs of the own vehicle trajectory prediction task head, and the own vehicle trajectory prediction task head is used to output the own vehicle intention prediction result and the own vehicle trajectory planning result; Figure 3 In the figure, Q0, Q1, Q2, and Q3 are the query networks of the four task heads respectively. The outputs of Q1, Q2, and Q3 and the output of the large model are used as the input features of Q0. In this way, the output features of the large model can be encoded in one layer and then combined with the original features to form the input of Q0. The cascade form can effectively improve the interpretability of the model and the debugging efficiency of each module.

[0038] Furthermore, the self-vehicle trajectory planning result output by the self-vehicle trajectory prediction task head is verified by the safety decision posterior network (mainly to verify whether the trajectory is compliant, collision-free, safe and comfortable). If the safety conditions are met, the self-vehicle trajectory planning result and the self-vehicle intention prediction result are sent to the vehicle control module for generating vehicle control instructions; if the safety conditions are not met, a backup trajectory planning result is generated, and the backup trajectory planning result and the self-vehicle intention prediction result are sent to the vehicle control module for generating vehicle control instructions.

[0039] Specifically, the safety decision posterior network evaluates whether the vehicle trajectory planning result meets the safety condition by means of threshold judgment.

[0040] Specifically, if the safety conditions are not met, the self-vehicle following trajectory in the lane is generated as a backup trajectory through sampling to strictly ensure vehicle safety and ensure that when the model performance degradation occurs, the vehicle can degenerate to following in the lane to ensure safety.

[0041] In this embodiment, in addition to outputting the vehicle's trajectory planning results, the intelligent driving model also outputs the vehicle's intention prediction results to facilitate human-computer interaction. For example, the cockpit actively broadcasts the behavior of the vehicle ahead based on the intelligent driving behavior and takes actions such as turning on the lights and adjusting the suspension lateral support.

[0042] like Figure 3As shown, specifically, each of the task heads is connected to an Adapter module for fine-tuning the model. In this embodiment, there are 5 Adapter modules, wherein Adapter1, Adapter2, and Adapter3 are connected to the back of Q1, Q2, and Q3 respectively; Adapter4 and Adapter5 are located behind two branches of the output feature of Q0 respectively. Q0 outputs the self-vehicle combined output feature, one of which passes through Adapter4 and obtains the self-vehicle trajectory planning result point by point through adaptive regression coding, and the other passes through Adapter5 to obtain the self-vehicle intention prediction result.

[0043] Specifically, the vehicle posture information includes: roll angle, roll angular velocity, roll angular acceleration; yaw angular velocity, yaw angular acceleration; lateral velocity, lateral acceleration, lateral jerk; longitudinal velocity, longitudinal acceleration, longitudinal jerk; pitch angle, pitch angular velocity, pitch angular acceleration.

[0044] Specifically, the vehicle control module includes:

[0045] The lateral and longitudinal tracking control module is used to control the vehicle's braking, driving and steering;

[0046] The exterior lighting control module is used to control the high and low beam lights and turn signals, etc.

[0047] Human-computer interaction module, used to realize voice and visual interaction between the cockpit and the user;

[0048] Seat control module, which controls seat side bolsters and seat belt tensioning;

[0049] Suspension control module, used to adjust the suspension height and hardness.

[0050] Specifically, the vehicle control instructions include driving instructions, braking instructions, steering instructions, lighting control instructions, human-computer interaction control instructions, seat adjustment instructions, and suspension adjustment instructions.

[0051] A second aspect of this embodiment provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the intelligent driving closed-loop control method are implemented.

[0052] A third aspect of this embodiment provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the intelligent driving closed-loop control method are implemented.

[0053] This embodiment integrates the full amount of heterogeneous input features in a closed-loop form, including historical trajectory planning, environmental perception results, local mapping, navigation information, and dynamic data such as vehicle posture, and uses a learnable full-input precoder to encode the input features and generate standardized inputs suitable for pre-trained large language model processing. Then, the full amount of features are extracted through the pre-trained large language model with frozen parameters to generate shared Key-Value pairs for use by multiple task heads. The task head includes multiple cascade modules, which output the prediction results at the intent level and trajectory level through an independent trainable Query network, where the trajectory-level output needs to be verified by the safety decision posterior network to ensure its safety and availability, and generate a backup trajectory when necessary. The verified trajectory and intention are sent to the vehicle control module for the control of functions such as steering, driving, braking, lighting and suspension, and these output information are fed back to the model as closed-loop input to form a dynamic closed-loop optimization mechanism. This implementation method not only improves the accuracy and safety of decision-making planning by integrating multi-source information, optimizing feature extraction, and introducing safety verification and closed-loop feedback, but also enhances the robustness and adaptability of the system in complex dynamic environments.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent driving closed-loop control method based on a pre-trained large model, characterized in that: The following steps are involved: S1, obtaining full heterogeneous input data during vehicle driving, wherein the full heterogeneous input data includes closed-loop input data, environment perception results of previous N frames, local mapping results of the current frame, and navigation information of the current frame; S2, feature encodes all heterogeneous input data to generate feature data suitable for processing by the pre-trained large model; S3, input the encoded feature data into the pre-trained large model with frozen parameters for feature extraction, and output key-value pairs; S4, builds multiple task heads in cascade form to share the key-value pairs of the large model output. Each task head holds an independent trainable query network, which is used to generate intent-level and trajectory-level outputs respectively; S5, generates vehicle control instructions based on the intention-level output and trajectory-level output to control the vehicle's driving state, and feeds back the trajectory-level output, the vehicle's current control state and posture information as closed-loop input data to the next frame of input data of the large model, forming a closed-loop feedback.

2. The intelligent driving closed-loop control method based on a pre-trained large model as claimed in claim 1, characterized in that: In step S2, after the input data from different sources are independently feature encoded and information compressed, an ensemble learning algorithm is used to combine them into full input features.

3. The intelligent driving closed-loop control method based on a pre-trained large model as claimed in claim 1, characterized in that: In step S4, the multiple task heads in cascade form include two layers of task heads, the first layer of task heads includes the other vehicle lateral intention prediction task head, the other vehicle longitudinal intention prediction task head and the scene recognition task head, and the second layer of task heads is the own vehicle trajectory prediction task head; the outputs of the other vehicle lateral intention prediction task head, the other vehicle longitudinal intention prediction task head and the scene recognition task head and the key-value pairs of the large model output are used as the input of the own vehicle trajectory prediction task head, and the own vehicle trajectory prediction task head is used to output the own vehicle intention prediction result and the own vehicle trajectory planning result.

4. The intelligent driving closed-loop control method based on a pre-trained large model as claimed in claim 3, characterized in that: The self-vehicle trajectory planning result output by the self-vehicle trajectory prediction task head is verified by the safety decision posterior network. If the safety condition is met, the self-vehicle trajectory planning result and the self-vehicle intention prediction result are sent to the vehicle control module for generating vehicle control instructions; If the safety conditions are not met, a fallback trajectory planning result is generated, and the fallback trajectory planning result and the vehicle intention prediction result are sent to the vehicle control module for generating vehicle control instructions.

5. The intelligent driving closed-loop control method based on a pre-trained large model as claimed in claim 4, characterized in that: The safety decision posterior network evaluates whether the vehicle trajectory planning result meets the safety conditions by means of threshold judgment.

6. The intelligent driving closed-loop control method based on a pre-trained large model as claimed in claim 4, characterized in that: If the safety conditions are not met, the self-vehicle following trajectory in the lane is generated as a backup trajectory through sampling.

7. The intelligent driving closed-loop control method based on a pre-trained large model as claimed in claim 1, characterized in that: Each of the task heads is connected to an Adapter module for fine-tuning the model.

8. The intelligent driving closed-loop control method based on a pre-trained large model as claimed in claim 1, characterized in that: The vehicle control instructions include driving instructions, braking instructions, steering instructions, lighting control instructions, human-computer interaction control instructions, seat adjustment instructions, and suspension adjustment instructions.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the intelligent driving closed-loop control method as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the intelligent driving closed-loop control method as described in any one of claims 1 to 8 are implemented.

Images